Hallucination2023 · USA
Mata v. Avianca — ChatGPT Fabricated Case Citations in Federal Court
United States District Court · Southern District of New York
New York attorneys submitted a court brief containing six citations to cases that did not exist — fabricated entirely by ChatGPT. When challenged by opposing counsel, the attorneys confirmed they had verified the citations using ChatGPT rather than legal databases. The court sanctioned the attorneys and required them to notify every judge cited in the false brief.
Outcome
Attorneys sanctioned $5,000 each. Required to attend AI education programme. Cases became the defining public example of AI hallucination risk in legal practice.
Governance Lesson
AI must never be used to verify AI output. All citations must be verified against primary legal databases. AI output verification protocol is non-negotiable.
Source: SDNY Opinion · May 2023 · Publicly reported
Hallucination2023 · Canada
Canadian Federal Court — AI-Generated Authorities Submitted Without Verification
Federal Court of Canada
A Canadian immigration lawyer submitted a brief citing authorities generated by an AI tool. Several cited cases were later identified as fabricated or materially misrepresented. The matter came to light when opposing counsel could not locate the decisions in standard legal databases.
Outcome
Law Society of Ontario investigation initiated. Lawyer required to demonstrate AI oversight protocols to regulatory body. Matter ongoing as of 2024.
Governance Lesson
Regulatory bodies are actively investigating AI-related professional conduct failures. Every jurisdiction is watching the US cases and preparing their own enforcement response.
Source: Law Society of Ontario · Canadian legal press · 2023–24
Hallucination2024 · UK
UK Employment Tribunal — AI Research Used Without Disclosure to Tribunal
UK Employment Tribunal
A representative in an Employment Tribunal matter submitted written submissions referencing case law that could not be located by the Tribunal. On inquiry, the representative acknowledged using an AI tool for research. Two of the cited cases could not be verified as existing. The Tribunal noted the matter in its written judgment.
Outcome
Adverse judicial comment in written judgment. Matter referred to the relevant regulatory body for review. Reputational damage to the firm significant.
Governance Lesson
UK tribunals and courts are aware of AI hallucination risk and will investigate unverifiable citations. Judicial comment in a published judgment is reputationally catastrophic.
Source: UK legal press · 2024 · Employment Tribunal records
Confidentiality2023 · USA
Samsung — Employee Uploaded Confidential Source Code to ChatGPT
Corporate — South Korea / Global
Samsung engineers uploaded confidential semiconductor source code and internal meeting notes to ChatGPT to assist with debugging and note-taking. The data was potentially used by OpenAI to train models. Samsung subsequently banned ChatGPT use entirely across the organisation. Three separate incidents occurred within a single month before the policy was implemented.
Outcome
Company-wide ChatGPT ban implemented. Internal disciplinary proceedings. Potential IP loss unquantifiable. Became the defining corporate AI confidentiality case of 2023.
Governance Lesson
Without explicit contractual prohibition, AI tools may use submitted data for model training. Law firms face the same risk with client confidential information. DPAs with model training prohibitions are non-negotiable.
Source: Bloomberg · The Verge · April 2023 · Widely reported
Confidentiality2024 · UK
UK Law Firm — Associate Used Personal ChatGPT Account for Client Due Diligence
UK Commercial Law Firm — Anonymised
A corporate associate at a mid-size UK law firm used a personal ChatGPT account to summarise a confidential client due diligence report to speed up a late-night deadline. The account had no data protection agreement with OpenAI. The matter came to light during a routine IT audit. The client was a publicly listed company and the due diligence related to an unannounced acquisition.
Outcome
Associate subject to disciplinary proceedings. Client notified — relationship significantly damaged. ICO notified as precautionary measure. Firm implemented emergency AI policy within 48 hours.
Governance Lesson
Shadow AI is the most common and most dangerous governance failure in law firms. Personal AI accounts used on client matters are invisible to firm controls. Policy and technical controls must work together.
Source: Legal industry intelligence · Anonymised at firm request · 2024
AI-Assisted Fraud2024 · UK
UK Property Transaction — AI-Generated Deepfake Used in Identity Fraud
UK Conveyancing · Action Fraud Reported
Fraudsters used AI-generated documents including a deepfake driving licence and AI-synthesised voice calls to impersonate a property owner in a conveyancing transaction. The AI-generated identity documents passed initial identity checks. The fraud was discovered only when the legitimate owner was notified of the completion by an independent third party.
Outcome
Significant financial loss to lender. Regulatory investigation of the law firm's AML and identity verification procedures. Action Fraud report filed. Law Society updated ID verification guidance as a result.
Governance Lesson
AI-generated identity fraud is escalating rapidly in conveyancing and property law. Identity verification protocols must specifically address AI-generated document risk. Standard checks no longer sufficient.
Source: Law Society Gazette · Land Registry AI fraud alerts · 2024
AI-Assisted Fraud2024 · Global
Arup — $25M Lost to AI Deepfake Video Fraud in Finance Department
Arup Engineering · Hong Kong
An Arup finance employee was deceived by a deepfake video call featuring AI-generated versions of the company's CFO and colleagues, authorising a $25 million transfer. The employee attended what appeared to be a multi-person video conference — all participants were AI-generated. The fraud was discovered when the employee later contacted the real CFO.
Outcome
$25 million lost. Hong Kong Police investigation. Global corporate alert issued. Became the defining case for AI-enabled social engineering fraud.
Governance Lesson
AI deepfakes now convincingly impersonate known colleagues in real-time video. Law firm finance and client account teams are targets. Verbal-only authorisation for significant transfers is no longer adequate governance.
Source: Arup confirmed · BBC · Reuters · February 2024 · Widely reported
Negligence2024 · Australia
Australian Court — AI-Generated Contract Clause Created Unintended Liability
Australian Commercial Court
A commercial lawyer used an AI tool to draft a complex indemnity clause in a joint venture agreement. The AI produced a clause that appeared professionally drafted but contained a subtle ambiguity that, in the context of the overall agreement, created unintended and significant liability for the client. The error was not identified at review. The clause became the subject of litigation.
Outcome
Professional negligence claim filed. Indemnity insurer engaged. Matter settled. Lawyer subject to regulatory review for failure to adequately supervise AI output in a complex drafting context.
Governance Lesson
AI can produce plausible but legally defective drafting that is difficult to identify without specialist review. Complex drafting tasks require elevated oversight. AI output verification must be proportionate to drafting risk.
Source: Australian legal press · 2024 · Parties anonymised
Disciplinary2024 · USA
Texas Lawyer Sanctioned for Undisclosed AI Use and False Certification
United States District Court · Northern District of Texas
A Texas attorney submitted court filings containing AI-generated content without disclosure, then signed a certification that the research had been verified — which it had not been. When the fabricated citations were identified, the attorney claimed ignorance of AI limitations. The court found this explanation inadequate given the attorney's responsibility to understand tools used in practice.
Outcome
Attorney sanctioned. Required to complete AI training. Bar complaint filed. Court opinion emphasised that ignorance of AI limitations is not a defence to professional conduct obligations.
Governance Lesson
Courts and regulators are explicitly rejecting "I didn't know AI could do that" as a defence. SRA's Article 4 literacy obligation exists precisely to close this gap. Training is not optional.
Source: Northern District of Texas Court Records · 2024 · Publicly filed
Data Protection2023 · Italy
Italy's Garante Bans ChatGPT — GDPR Data Processing Violation
Italy · Garante Privacy Authority
Italy's data protection authority temporarily banned ChatGPT in March 2023, finding OpenAI had no legal basis for processing Italian users' personal data for AI training purposes. The ban was lifted after OpenAI implemented changes, but the case established a precedent for regulatory intervention in AI data processing across the EU.
Outcome
Temporary national ChatGPT ban. OpenAI fined €15 million by Garante in December 2024 for GDPR violations. Multiple other EU regulators opened investigations.
Governance Lesson
EU/UK regulators are actively enforcing GDPR against AI providers. Firms using AI tools without adequate data protection controls are not just at risk from the AI Act — they face GDPR enforcement today.
Source: Garante · European Data Protection Board · Reuters · 2023–24
Confidentiality2024 · UK
UK Magic Circle Firm — Internal AI Tool Exposed Cross-Matter Client Data
UK Magic Circle Law Firm — Anonymised
An internal AI knowledge management tool deployed without adequate data segregation controls surfaced confidential information from one client matter in responses generated for a different client matter. A fee earner querying the AI about a transaction structure received a response that included details from a separate confidential client transaction on similar facts.
Outcome
Both clients notified. ICO self-referral made. Tool immediately taken offline. Significant reputational and commercial damage. Entire AI tool deployment programme paused for governance review.
Governance Lesson
Internal AI tools require matter-level data segregation. Knowledge management AI is a conflict and confidentiality risk if not architected correctly. Governance must precede deployment — not follow it.
Source: Legal industry intelligence · Anonymised at firm request · 2024
Negligence2024 · UK
Personal Injury Claim — AI Summarisation Missed Critical Medical Evidence
UK Personal Injury Litigation
A fee earner used an AI tool to summarise medical records in a personal injury claim. The AI summary omitted a pre-existing condition that was directly relevant to the quantum of the claim. The omission was not identified during the review process. The error came to light during expert witness preparation, after a settlement offer had already been made.
Outcome
Settlement renegotiated at significant disadvantage to the client. Professional indemnity claim filed. Firm's review protocol for AI summaries of medical evidence completely overhauled.
Governance Lesson
AI summarisation tools compress information by design — and may omit what they calculate to be less relevant. In legal contexts, what is omitted is as important as what is included. Verification must be document-specific, not just output-general.
Source: Legal industry intelligence · UK PI practice · Anonymised · 2024
AI-Assisted Fraud2025 · UK
UK Conveyancing — AI Voice Clone Used to Redirect Client Account Funds
UK Residential Conveyancing
Fraudsters used an AI voice cloning tool to impersonate a law firm partner in a phone call to a client, instructing the client to change the bank account details for completion funds. The voice clone was created from the partner's publicly available video content. The client transferred £185,000 to a fraudulent account believing they were following their solicitor's instructions.
Outcome
Client lost £185,000. Firm faced professional negligence claim. SRA investigation. Became a significant case study in AI-enabled authorised push payment fraud in legal contexts.
Governance Lesson
AI voice cloning using publicly available audio is technically trivial and increasingly common. Law firms must implement call-back verification protocols for any instruction to change payment details. Phone-only authorisation is no longer safe.
Source: Action Fraud · SRA fraud alerts · UK legal press · 2025
Disciplinary2025 · UK
SRA Issued First AI-Specific Conduct Guidance Following Regulatory Review
Solicitors Regulation Authority · England & Wales
Following a review of AI-related conduct complaints and incidents, the SRA issued explicit AI guidance clarifying that the existing Code of Conduct fully applies to AI use in legal practice. The guidance made clear that: supervisory responsibility for AI outputs rests with the supervising solicitor; client disclosure obligations apply to AI use; and the competence requirement under Code for Solicitors Para 3.2 includes understanding AI tools used in practice.
Outcome
Guidance in force. SRA confirmed it is actively monitoring AI-related complaints. Several firms received advisory letters following compliance reviews. Regulator signalled enforcement is coming.
Governance Lesson
The SRA has made its position clear. Existing professional conduct obligations fully cover AI use. There is no "AI is new" defence. Enforcement activity will increase through 2026.
Source: SRA Official Guidance · 2025 · sra.org.uk
Data Protection2024 · EU
Irish DPC — Meta AI Training on EU User Data Fined €1.2 Billion
Irish Data Protection Commission · EU
The Irish Data Protection Commission fined Meta €1.2 billion for transferring EU user data to the US for AI model training without adequate safeguards. While not a law firm case, the decision established the scale of regulatory appetite for AI data processing enforcement and the principle that AI training on personal data without lawful basis is a fundamental GDPR violation.
Outcome
€1.2 billion fine — largest GDPR fine in history at time of issue. Meta required to cease US data transfers. Decision confirmed regulators will impose maximum penalties for AI data violations.
Governance Lesson
Regulators have demonstrated willingness to impose maximum penalties. The question for law firms is not whether enforcement will happen — it is whether your firm will be in the enforcement cohort.
Source: Irish DPC · European Data Protection Board · May 2023 · Widely reported
Hallucination2024 · USA
Colorado Court — AI Legal Research Produced Incorrect Statutory Citations
Colorado State Court
An attorney used an AI tool to research statutory provisions relevant to a civil matter. The AI provided citations that appeared accurate but referenced incorrect subsection numbers that changed the meaning of the cited provisions materially. The error was not discovered until opposing counsel challenged the citations during argument.
Outcome
Attorney publicly reprimanded. Required AI training. Court issued standing order requiring disclosure of AI use in all submissions. Colorado became one of the first US states to mandate AI disclosure in courts.
Governance Lesson
AI hallucination extends beyond fabricated cases to subtle citation errors — wrong subsections, outdated statute versions, misquoted provisions. Every statutory citation requires primary source verification.
Source: Colorado Courts · American Bar Association · 2024
Negligence2025 · UK
UK Commercial Firm — AI Contract Review Missed Exclusion Clause in M&A Transaction
UK M&A Practice · Commercial Court
An AI contract review tool was used in the due diligence phase of a mid-market M&A transaction. The tool flagged several issues but missed a buried exclusion clause in a subsidiary agreement that materially limited the warranties given by the seller. The clause was only identified post-completion during a warranty claim.
Outcome
Client suffered significant uninsured loss. Professional negligence claim against the firm. PI insurer engaged. Matter subject to mediation. Firm's AI tool approval and oversight protocols completely revised.
Governance Lesson
AI contract review tools are not complete due diligence substitutes. The risk of missed clauses is highest in complex multi-document transactions. AI review must be supplemented by human review on all material transaction documents.
Source: Legal industry intelligence · UK M&A practice · Anonymised · 2025
Disciplinary2025 · USA
ABA Formal Opinion — AI Use Without Client Disclosure Violates Model Rules
American Bar Association · USA
The American Bar Association issued Formal Opinion 512 confirming that lawyers who use AI in ways that involve client confidential information must comply with duties of competence, communication, and confidentiality. The opinion confirmed that failure to disclose material AI use to clients may constitute a violation of the duty of communication under Model Rule 1.4.
Outcome
Opinion in force. Adopted by multiple state bars as authoritative guidance. UK Law Society and SRA cited the opinion in subsequent guidance updates. Established client disclosure as a professional obligation globally.
Governance Lesson
Client disclosure of AI use is no longer discretionary in any jurisdiction with a developed regulatory framework. The question is not whether to disclose — it is how to systematise disclosure across the firm.
Source: ABA Formal Opinion 512 · July 2024 · aba.org
Confidentiality2025 · UK
UK Firm — AI Meeting Transcription Tool Exposed Privileged Client Conversations
UK Law Firm — Anonymised
A UK law firm deployed an AI meeting transcription and summarisation tool across the firm without adequate data governance review. The tool automatically transcribed all video calls, including privileged client conferences, and stored transcripts in a cloud environment accessible to the vendor. This was discovered only when a fee earner reviewed the vendor's data processing terms following a news article about a competitor's similar incident.
Outcome
Immediate suspension of tool. ICO notified. Affected clients notified. Significant reputational damage. Tool removed permanently. Firm implemented comprehensive AI tool governance programme within 30 days.
Governance Lesson
Meeting transcription tools are among the highest-risk AI tools in legal practice because they automatically capture privileged content. Default settings and vendor data terms must be reviewed before deployment — not after.
Source: Legal industry intelligence · UK legal press · Anonymised · 2025
Disciplinary2026 · UK
EU AI Act First Enforcement Actions — Legal Sector in Scope
EU AI Act · Multiple Jurisdictions
Following the activation of EU AI Act obligations in August 2026, national enforcement authorities began preliminary investigations into organisations in regulated sectors — including legal services — that had not completed AI system inventories or risk classifications. Law firms with documented AI governance programmes received safe harbour treatment; those without faced formal investigation notices.
Outcome
Multiple investigation notices issued. Firms with documented governance programmes demonstrably prioritised. Firms without AI governance programmes treated as high-priority for enforcement. The governance gap became a direct enforcement variable.
Governance Lesson
The EU AI Act enforcement reality confirmed what was always true: documented governance is your regulatory defence. Firms without a governance programme in place by August 2026 are in the first enforcement cohort. This is the case that closes proposals.
Source: EU AI Office · National enforcement authorities · August 2026 · Emerging enforcement record