Sponsored content from Hope & May
Artificial intelligence is now part of everyday work. AI tools help organisations work faster and more efficiently than ever. For many charities and businesses, the productivity gains are undeniable.
However, while AI is evolving at remarkable speed, data protection law has not changed nearly as quickly. For leaders, this creates an important challenge. The UK GDPR was not written with today’s generative AI systems in mind, yet organisations remain fully accountable for how personal data is collected, processed and protected. Simply because a tool is readily available does not mean it can be used without careful consideration.
For leaders, this means knowing where some of the key risks and responsibilities lie, and making sure the right questions are being asked. Below are some of the areas you may want to consider, or ask your organisation’s data protection lead or data protection officer to review, as AI becomes more widely used across your organisation.
Transparency
Transparency is one of the core principles of UK GDPR. Individuals have the right to understand how their personal data is being used. This becomes considerably more difficult when AI is involved.
Many AI systems operate as complex “black boxes”. Even the organisations developing them cannot always explain precisely how a particular output was produced or what weighting was given to different data sources.
If an organisation cannot clearly explain how an AI system processes personal information, can it truly meet its transparency obligations?
Leaders should be asking themselves whether their privacy information remains accurate if AI is introduced into existing processes. If employees, supporters, customers or beneficiaries would reasonably expect a human to make decisions, but AI is now playing a role, organisations may need to review their privacy notices and governance arrangements.
Decision-making
Under UK GDPR, individuals have specific protections where decisions are made solely by automated means and those decisions have legal or similarly significant effects.
While many organisations still use AI as an assistant rather than a decision-maker, the line is blurring. Recruitment shortlisting, fraud detection, service eligibility, supporter profiling and risk assessments are all areas where AI may begin to influence outcomes.
Even where a human signs off the final decision, leaders should consider whether that human is genuinely exercising independent judgement or simply accepting an AI recommendation without meaningful review.
Human oversight should be real, informed and capable of challenging AI outputs – not merely a rubber-stamping exercise.
Bias
AI systems learn from existing information, much of which reflects historical human behaviour and societal bias. If training data contains bias, AI can reinforce it. If data is incomplete, AI can produce inaccurate or misleading conclusions.
For organisations making decisions about people, these risks cannot be ignored. A flawed AI-generated recommendation could influence recruitment, funding decisions, service delivery or supporter engagement in ways that are difficult to detect without careful monitoring.
Ultimately, responsibility for those decisions rests with the organisation – not the software provider.
Accuracy
The UK GDPR requires personal data to be accurate and, where necessary, kept up to date. Generative AI, however, is capable of producing entirely convincing but factually incorrect information – a phenomenon often referred to as “hallucination”.
If organisations rely on AI-generated summaries, profiles or recommendations without verification, inaccurate information could quickly become embedded within business processes.
For leaders, this reinforces an important principle: AI should assist professional judgement, not replace it.
Who is accountable?
Perhaps the biggest misconception surrounding AI is that responsibility somehow shifts to the technology provider. That may not be the case as AI systems may not have a legal personality.
In most cases, they cannot be held accountable for data protection breaches, unlawful processing or poor governance. The legal responsibility remains firmly with the organisation deciding to use the technology.
Boards, trustees and senior leadership teams should therefore view AI governance as a business risk rather than simply an IT issue. Policies, staff training, procurement processes and supplier due diligence all have an important role to play.
Special category data deserves extra protection
Perhaps the greatest area of concern is the use of AI with special category data. Information relating to health, ethnicity, religious beliefs, political opinions, trade union membership, sexual orientation or biometric data receives enhanced protection under UK GDPR for good reason.
Uploading this information into publicly available AI tools, or using unassessed AI to analyse or generate conclusions about individuals based upon special category data, presents significant risks.
Beyond questions of security and confidentiality, organisations must also consider whether the processing is lawful, fair and proportionate.
For many organisations, the safest approach is straightforward: avoid using publicly available generative AI tools with special category personal data altogether unless there is a clearly defined legal basis, appropriate safeguards and a thorough assessment of the associated risks.
AI needs governance
Artificial intelligence requires governance. Leaders should ensure they understand where AI is being used across their organisation, what personal data may be involved, whether appropriate policies are in place, and how decisions are being monitored and challenged.
The organisations that will benefit most from AI will be those that adopt it most responsibly.
As with any emerging technology, good governance is not a barrier to innovation—it is what allows innovation to happen safely, ethically and with confidence.