Compliance Measures for a SQL-Optimizing Language Model Under the EU AI Act


Does the development and deployment of a Language Model acting as a proxy to interpret and optimize SQL queries for better performance on database engines, which may potentially be employed by clients within the European Union and handle various types of data, including personal and sensitive information, and whose optimization could influence database query outcomes, indirectly impacting decision-making processes, fall within the regulatory scope of the EU's AI Act, and if so, what are the necessary compliance measures, specifically in the areas of transparency, accuracy, and human oversight?


The development and deployment of a Language Model (LM) for SQL query optimization likely falls under the EU's AI Act, as it can generate outputs affecting databases, which is considered an "AI system." If you're developing or employing such a system in any professional capacity within the EU, you would be recognized as the "provider" or "deployer" respectively, holding specific responsibilities for compliance. Key compliance areas include ensuring transparency about the system's purpose, maintaining accuracy, and implementing human oversight mechanisms. The level of compliance required correlates with the risk level associated with how the LM's optimizations might impact decisions based on the database query outcomes. Providers and deployers must ensure that the system performs in line with its intended purpose as outlined by the Act.

Legal trace

‘‘artificial intelligence system’ (AI system) means a machine-based system that is designed to operate with varying levels of autonomy and that can, for explicit or implicit objectives, generate outputs such as predictions, recommendations, or decisions, that influence physical or virtual environments; (Article 3(1))

The language model (LM) designed for optimizing SQL queries falls under this definition, as it autonomously produces outputs (in this case, optimized queries) that can have substantial impacts on virtual environments, like databases.

‘provider’ means a natural or legal person, public authority, agency or other body that develops an AI system or that has an AI system developed with a view to placing it on the market or putting it into service under its own name or trademark, whether for payment or free of charge; (Article 3(2))

For the LM in question, the developer or entity behind its creation, intending to market or use the LM, would be considered the provider under the Act. This holds true regardless of whether the service is paid or free.

‘deployer means any natural or legal person, public authority, agency or other body using an AI system under its authority except where the AI system is used in the course of a personal non-professional activity; (Article 3(4))

Should clients within the EU employ the LM in a professional capacity, they would be classified as deployers under the AI Act, coming with specific statutory duties mandated by the Act.

‘intended purpose’ means the use for which an AI system is intended by the provider, including the specific context and conditions of use, as specified in the information supplied by the provider in the instructions for use, promotional or sales materials and statements, as well as in the technical documentation; (Article 3(12))

The LM must clearly define its intended purpose, which influences the regulatory requirements it will be subject to. For the LM in discussion, optimizing database queries is considered an intended purpose that falls within the purview of the AI Act.

‘risk’ means the combination of the probability of an occurrence of harm and the severity of that harm; (Article 3(1a))

The risk associated with the LM, defined as the probability and severity of potential harm, will inform the level of compliance measures necessary, especially in the domains of transparency, accuracy, and human oversight.

‘performance of an AI system’ means the ability of an AI system to achieve its intended purpose; (Article 3(18))

The performance of the LM, notably its capacity to effectively optimize SQL queries, should align with the AI Act’s standards. This implies an obligation on the provider and deployer to guarantee that the system performs adequately and reliably.