AI in IT: GDPR-Related Issues

Question

What are the problems with AI in IT based on GDPR ?

Executive Summary

Addressing GDPR issues with AI in the IT sector is critical due to the rigid data protection requirements. The following summary outlines the major challenges and obligations entrepreneurs need to tackle:

  • Personal Data and Transparency: AI must comply with GDPR’s broad definition of personal data and the need for transparent processing, creating a challenge due to the complexity of AI systems.
  • Controller and Processor Responsibilities: Both AI system designers (controllers) and those operating the systems (processors) are obliged to show GDPR compliance in technical and organizational measures.
  • Consent and Data Subject Rights: AI systems must have mechanisms to obtain and demonstrate consent from users and support their rights in data management, despite AI’s inherent complexity in providing such transparency.
  • Automated Decision-Making and Design Protocols: AI faces restrictions in automated decision-making, requiring options for human oversight. Furthermore, systems must integrate data protection by design, including encryption and pseudonymization.
  • Legality and Sensitive Data: Processing must be lawful, with AI operations often needing to rely on contracts or legitimate interests, and must be particularly cautious when handling sensitive categories of personal data.

Assumptions

  1. Scope of AI Applications: Assume that AI applications in IT cover data analytics, predictive modeling, and automated decision-making systems, which are typical uses with prominent GDPR implications.

  2. Specific GDPR Concerns: Assume the user is interested in a broad overview of GDPR issues, including but not limited to consent requirements, automated decision-making, profiling, and data subject rights such as access and rectification.

  3. AI Operational Context: Assume that AI systems are operated by organizations that could function both as data controllers and processors, necessitating a comprehensive GDPR compliance strategy.

  4. Nature of Issues: Assume the user seeks information on legal and compliance issues that AI might face around processing personal data under GDPR.

Legal trace

The Definition and Principle Framework and Their Implications for AI

’personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person; Article 4(1)

This definition underscores the broadness of what constitutes personal data within AI systems, which can ingest diverse data types, potentially revealing personal identifiers. This necessitates AI developers to carefully consider the data their systems process, ensuring compliance with GDPR’s broad interpretation of personal data.

Personal data shall be processed lawfully, fairly and in a transparent manner in relation to the data subject (‘lawfulness, fairness and transparency’); Article 5(1)(a)

This principle addresses the need for AI systems to manifest their processing activities in an understandable manner. Transparency issues emerge due to AI systems’ complexity, highlighting the necessity for systems to clearly inform data subjects of how their data is being used and processed.

Controller and Processor Roles in AI Systems

Taking into account the nature, scope, context and purposes of processing as well as the risks of varying likelihood and severity for the rights and freedoms of natural persons, the controller shall implement appropriate technical and organisational measures to ensure and to be able to demonstrate that processing is performed in accordance with this Regulation. Article 24(1)

AI system designers functioning as controllers are laden with the responsibility to implement and demonstrate compliance-aligned processing procedures, establishing the need for proactive technical and organizational measures reflecting the GDPR’s stringent requirements.

Where processing is to be carried out on behalf of a controller, the controller shall use only processors providing sufficient guarantees to implement appropriate technical and organisational measures in such a manner that processing will meet the requirements of this Regulation and ensure the protection of the rights of the data subject. Article 28(1)

Processors, too, are under obligation to uphold GDPR’s standards, necessitating AI systems to be built and operated with inbuilt GDPR compliance measures. This calls for a demonstration by AI developers and processors of their ability to protect data subject rights adequately.

Where processing is based on consent, the controller shall be able to demonstrate that the data subject has consented to processing of his or her personal data. Article 7(1)

Consent is pivotal, and AI systems require mechanisms to demonstrate that data subjects have been duly informed and have expressed consent. This complexifies the consent management process, especially in AI systems where decision rationales are less transparent.

The controller shall take appropriate measures to provide any information referred to in Articles 13 and 14 and any communication under Articles 15 to 22 and 34 relating to processing to the data subject in a concise, transparent, intelligible and easily accessible form, using clear and plain language… Article 12(1)

Rights such as access, rectification, erasure, and portability impose on AI systems the need to integrate user-friendly mechanisms for data subjects to manage their data. AI’s complexity must be reconciled with GDPR’s mandate for clarity and accessibility.

Automated Decision-Making, Profiling, and Data Protection by Design

The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling… Article 22(1)

Automated decision-making, a fundamental AI functionality, faces direct stipulations under GDPR, where people have the right to reject automated decisions without human oversight. This affects AI system design, requiring pathways for human intervention and review.

Taking into account the state of the art…the controller shall…implement appropriate technical and organisational measures, such as pseudonymisation, which are designed to implement data-protection principles… Article 25(1)

Data protection by design demands AI systems to be fashioned from their inception with compatible data protection measures. AI applications must hence adopt encryption, pseudonymization, and other strategies that ensure compliant data handling.

Legality of AI Data Processing and the Impact on Special Data Categories

Processing shall be lawful only if…the processing is necessary for the performance of a contract…or necessary for the purposes of the legitimate interests… Article 6(1)(b), (f)

AI systems that process data without explicit consent must ensure their operations fall under alternative legal bases such as contracts or legitimate interests. Striking a balance between these interests and the rights of the individuals remains cardinal and challenging.

Processing of personal data revealing racial or ethnic origin…shall be prohibited. Article 9(1)

AI systems must navigate the thorny terrain of handling sensitive information, as GDPR places strict prohibitions on processing certain data categories. Any AI operation within this realm must be extremely cautious, obtaining explicit consent or aligning with specific exemptions under GDPR.


In essence, the GDPR poses significant challenges for AI in IT, demanding rigorous design, operation, and governance to ensure compliance. From defining what constitutes personal data to implementing consent and data subject rights, accommodating automated decision-making, and handling sensitive data, AI systems are expected to uphold the rights and freedoms of individuals with respect to data protection, which often entails navigating complex technical and legal roadmaps.

PDF Repository

We have searched through the PDF repository of ECJ rulings, European Data Protection Board guidelines, and other documents to provide this supplemental answer.

Details

In furtherance of our previous discussion on the challenges of AI in IT environments under the GDPR regime, we now delve deeper into specific aspects and implications of GDPR's provisions on profiling, automated decision-making, and individual rights. The following legal excerpts will highlight pertinent considerations for AI development and use within the prescribed legal framework.

Legal trace

Expanded Interpretation of Profiling and Its Sector-Specific Implications

Profiling and automated decision-making are used in an increasing number of sectors, both private and public. Banking and finance, healthcare, taxation, insurance, marketing and advertising are just a few examples of the fields where profiling is being carried out more regularly to aid decision-making. Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679, page 5

The prevalence of profiling in sectors like banking, finance, and healthcare illustrates the expansive reach of AI systems that needs to align with GDPR’s standards. This context clarifies for clients which industries are significantly impacted by GDPR regulations and underscores the importance of compliance across diverse applications of AI.

Controllers can carry out profiling and automated decision-making as long as they can meet all the principles and have a lawful basis for the processing. Additional safeguards and restrictions apply in the case of solely automated decision-making, including profiling, defined in Article 22(1). Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679, page 9

This statement lays out the foundational requirement for AI systems regarding lawful bases for profiling and automated decision-making. It links directly to our original analysis, emphasizing the necessity for AI systems to not only establish a lawful basis but also adhere strictly to GDPR’s principles and incorporate requisite safeguards.

Constraints and Rights in Profiling and Special Category Data

Profiling can create special category data by inference from data which is not special category data in its own right but becomes so when combined with other data. Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679, page 15

The transformation of standard data into a special category through AI profiling processes implicates additional GDPR obligations and places constraints on AI systems. This insight builds upon our initial analysis by delineating the careful handling required when profiling leads to the generation of sensitive data categories.

Human Oversight and the Rule of Law in AI Decisions

The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling… Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679, page 19

Here, we see GDPR’s stipulation for human oversight in AI decision-making, reinforcing our original answer’s point on automated decision-making. It highlights the protection afforded to individuals against decisions made without human intervention, an essential consideration for ethical and legally compliant AI operations.

Addressing Bias, Protecting Children, and Ensuring Compliance Mechanisms

Controllers should carry out frequent assessments on the data sets they process to check for any bias, and develop ways to address any prejudicial elements, including any over-reliance on correlations. Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679, page 28

The predisposition towards bias in AI systems necessitates active assessment and strategic measures to mitigate potential discrimination. This quote complements our main analysis’s exploration of GDPR’s requirements by detailing the ongoing responsibilities AI controllers have to monitor and correct for biases within the data sets they utilize.

Synthesizing Insights for a Robust GDPR-Aligned AI Strategy

These excerpts, distilled from extensive GDPR-focused documents, enhance our understanding of the regulation’s reach, the detailed obligations it places on AI systems, and the rights it affords to individuals. They collectively inform on crafting a comprehensive GDPR-aligned strategy for AI in IT, emphasizing sector-specific implications, lawful bases, transparency, human oversight, and the proactive steps necessary to ensure bias-free, ethical AI systems.