Artificial Intelligence (“AI”) has become increasingly pervasive in everyone’s daily lives – with its rapidly improving capabilities, it is becoming increasingly difficult for people to use AI safely in some shape or form in their personal or work lives.
This is the case in the legal sector as well but the use of AI has been making headlines for all the wrong reasons. Here are some examples that have taken place in 2025 itself:
- A US lawyer was sanctioned for submitting a filing drafted using ChatGPT which contained a non-existent court case[1]. The Utah Court of Appeal ordered him to pay the respondent’s attorney fees for the petition and hearing, refund his client’s fees for the time used to prepare the filing and attend the hearing and donate $1,000 to a legal non-profit organisation.
- A Canadian lawyer was reprimanded for using non-existent cases and misreadings of real cases[2]. She avoided charges on contempt of court after promising the judge she will take courses on legal ethics and artificial intelligence and not bill her client for work related to this matter[3].
- An Australian lawyer was referred to a state legal complaints commission for submitting case citations which did not exist[4].
All these instances have a common theme – the lack of human oversight over the AI outputs. With how AI has been developing and its capabilities, it is understandable for legal professionals to want to use it to reduce their time burden.
This article aims to provide some basic tips to lawyers and other professionals on use of AI platforms including large language models like ChatGPT, Gemini and Claude.
Know how AI actually works to be able to use it safely
To be able to use AI effectively and correctly, lawyers should first know how the AI works and what it is capable for. It is important for people to understand that AI does not create content out of thin air. Instead, AI models have been trained on massive databases of content and they learn to create content by making predictions by finding patterns in the data that it has been trained on.
Understanding AI’s predictive nature is crucial for lawyers because it directly impacts professional competence requirements. Unlike traditional legal research tools that retrieve existing information, AI generates responses based on statistical patterns in training data. This means AI cannot distinguish between authoritative legal precedent and unreliable sources. For instance, it may treat a blog piece and a Supreme Court decision as equally valid patterns to replicate. For lawyers, this fundamental limitation means AI output must always be treated as a starting point requiring independent verification, never as a substitute for legal expertise or authoritative research.
Review and verify thoroughly
All examples provided in the introduction have one common theme: submitting non-existent precedents. Courts worldwide are responding with specific guidance – Australia’s New South Wales Supreme Court recently issued comprehensive practice directions that provide a roadmap for safe AI implementation[5].
This illustrates the risk of ‘hallucination’ when using an AI model. Hallucinations are responses by an AI model which are fabricated but appear authentic. These can be caused due to a variety of factors including: insufficient training data, incorrect assumptions made by the model, or biases in the data used to train the model.
A Google Cloud resource explains further[6]:
“An AI model may struggle to accurately understand real-world knowledge, physical properties, or factual information. This lack of grounding can cause the model to generate outputs that, while seemingly plausible, are actually factually incorrect, irrelevant, or nonsensical. This can even extend to fabricating links to web pages that never existed.
An example of this would be an AI model designed to generate summaries of news articles may produce a summary that includes details not present in the original article, or even fabricates information entirely.”
This makes it imperative for all output to be thoroughly reviewed and verified. You should ask the AI model to provide you with relevant sources and links. If it doesn’t, then you should conduct your own independent research to verify the research provided by the AI model.
You can also cross-question the AI model to confirm accuracy of segments of the output and more often than not, it will admit that it has no sources for some of the information that was provided in the output. Do not accept information provided by an AI model at face-value without conducting your own independent research.
Enhancement of input versus creating from scratch
This tip ties into Tip #2 above. With the risk of hallucinations, it is better to use AI as an enhancer of work/as a review stage in your work process as opposed to using AI to draft/research from scratch.
You can use AI to correct grammar, re-structure your writing or use better terminologies (all use should be subject to your verification of the AI model’s edits). AI should be used as an additional layer of check in your process as opposed to replacing the entire process of human intervention and review.
Protecting client confidentiality
Client confidentiality is a cornerstone of legal practice, and AI usage presents significant risks to this fundamental duty. Most public AI platforms like ChatGPT, Gemini, and Claude (some of these may not use your data to train models depending on your subscription type[7]) retain user inputs and may use them to train future models, potentially exposing sensitive client information.
The key confidentiality risks include data retention, where inputs may be stored indefinitely on AI company servers, and the possibility that your client’s information could become part of the model’s training data. Additionally, AI companies may be subject to data breaches or government requests, and there’s a risk of cross-contamination where client information might inadvertently appear in responses to other users.
Often, professional responsibility rules require lawyers to take reasonable measures to protect client information, and this duty generally extends to all technology used in legal practice. This makes Tip#1 essential – you need to know how the AI model that you are proposing to use works and its Terms of Service.
To maintain confidentiality, lawyers should never input client names, case-specific facts, privileged communications, or confidential business information into AI systems. Instead, queries should be reframed using hypotheticals and generic scenarios rather than actual client situations. It might also be prudent to obtain informed consent from clients in certain situations with respect to the use of AI in relation to their work.
Be aware of AI bias
AI bias presents distinct challenges from hallucinations. While hallucinations involve fabricated information, bias involves skewed perspectives embedded in AI training data that can influence legal analysis and reasoning.
The US Copyright Office’s Report on Copyright and Artificial Intelligence brilliantly illustrates how AI fills gaps with built-in assumptions[8]:
The Report states:
“This prompt describes the subject matter of the desired output, the setting for the scene, the style of the image, and placement of the main subject. The resulting image reflects some of these instructions (e.g., a bespectacled cat smoking a pipe), but not others (e.g., a highly detailed wood environment). Where no instructions were provided, the AI system filled in the gaps.
For instance, the prompt does not specify the cat’s breed or coloring, size, pose, any attributes of its facial features or expression, or what clothes, if any, it should wear beneath the robe. Nothing in the prompt indicates that the newspaper should be held by an incongruous human hand.”[9]
There are different types of biases that may affect legal work:
- Cultural and demographic bias: AI models trained predominantly on data from specific populations may not accurately reflect diverse legal contexts or client experiences. This is particularly problematic in areas like employment discrimination, family law, or criminal defence.
- Jurisdictional bias: AI may favour legal principles from jurisdictions better represented in training data (which is often US/UK/EU law) over Indian or other jurisdictional laws. For instance, if an AI model is asked to draft an employment policy, it may draft one based on US employment laws overlooking stronger protections for employees in the lawyer’s jurisdiction (for instance, the Netherlands[10]).
- Temporal bias: AI training data has cutoff dates which means that it is possible that recent legal developments, new statutes, or evolving case law may not be reflected in outputs.
- Source bias: As discussed above in Tip#1, AI cannot distinguish between authoritative legal sources and commentary – it may give equal weight to a Supreme Court opinion and a blog post discussing that opinion.
Hence, when reviewing output, always review and question the output for possible biases and unstated assumptions that the AI model might have incorporated.
Creating organisation-wide AI policy
Recognising these individual responsibilities is essential, but addressing AI risks effectively also requires systemic approaches at the firm or organisational level. With the spread of AI, firms should look to introduce AI policies and enforce the policies firmwide. This will give partners/supervising lawyers confidence about the manner of the use of AI in the work product that they have been asked to review by junior lawyers and will reduce their burden in having to verify the briefs for hallucinations, etc.
Having a policy also ensures that a standard of AI usage is established across the organisation which also provides clients with a level of comfort and transparency. Implementing these policies effectively requires the same systematic approach that makes time management successful – clear processes, consistent application, and regular review[11].
An AI policy should specify what types of information can and cannot be uploaded to AI models and identify use cases where AI is prohibited.
American Bar Association’s ethics guidance on the use of AI
In 2024, the American Bar Association’s (“ABA”) Standing Committee on Ethics and Professional Responsibility published its first formal opinion on the use of generative AI (“GAI”) in the field of law[12].
The ABA’s summary outlined the following rules in the ABA Model Rules of Professional Conduct that lawyers should be mindful of[13]:
- Model Rule 1.1 (Competence): This obligates lawyers to provide competent representation to clients and requires they exercise the “legal knowledge, skill, thoroughness and preparation reasonably necessary for the representation.” In addition, the model rule states lawyers should understand “the benefits and risks associated” with the technologies used to deliver legal services to clients.
- Model Rule 1.6 (Confidentiality of Information): Under this model rule, a lawyer using GAI must be cognizant of the duty to keep confidential all information relating to the representation of a client, regardless of its source, unless the client gives informed consent. Other model rules require lawyers to extend similar protections to former and prospective clients’ information.
- Model Rule 1.4 (Communications): This model rule addresses lawyers’ duty to communicate with their clients and builds on lawyers’ legal obligations as fiduciaries, which include “the duty of an attorney to advise the client promptly whenever he has any information to give which it is important the client should receive.” Of particular relevance to GAI, Model Rule 1.4(a)(2) states that a lawyer shall “reasonably consult” with the client about the means by which the client’s objectives are to be accomplished.
- Model Rule 1.5 (Fees): This rule requires a lawyer’s fees and expenses to be reasonable and includes criteria for evaluating whether a fee or expense is reasonable. The formal opinion notes that if a lawyer uses a GAI tool to draft a pleading and expends 15 minutes to input the relevant information into the program, the lawyer may charge for that time as well as for the time necessary to review the resulting draft for accuracy and completeness. But, in most circumstances, the lawyer cannot charge a client for learning how to work a GAI tool.
While this opinion applies specifically to US practitioners, it still provides a useful steer for lawyers in jurisdictions that do not have specific guidance published on the ethical use of AI.
Conclusion
As the examples at the start of the article show, the risk of using unverified information generated using an AI model is extremely high. As AI becomes increasingly sophisticated and accessible, the temptation to rely on it will only grow and this risk will only get higher.
However, the legal profession’s relationship with AI need not be adversarial. The aim is to utilise its benefits to deliver better and effective service to clients while maintaining professional standards.
As the technology continues to evolve, so too must our approaches to using it ethically and effectively. Staying current with these developments requires active engagement with the legal technology community through strategic professional networking[14].
The lawyers who fall foul of ethical guidelines are not necessarily those who avoid AI entirely, but those who fail to exercise the professional judgment that has always been at the heart of legal practice. In the age of AI, that judgment has never been more critical.
[1] Maya Yang, ‘Utah lawyer sanctioned for using ChatGPT to draft court filing with fake case’. The Guardian, 31 May 2025. https://www.theguardian.com/us-news/2025/may/31/utah-lawyer-chatgpt-ai-court-brief
[2] ‘Canadian lawyer uses AI to draft fake cases, faces contempt’. NDTV, 25 May 2025. https://www.ndtv.com/world-news/canadian-lawyer-uses-ai-to-draft-fake-cases-faces-contempt-8394213
[3] Gail J. Cohen, ‘Toronto lawyer avoids contempt charges for AI-generated fake citations’. Law.com, 22 May 2025. https://www.law.com/legaltechnews/2025/05/22/toronto-lawyer-avoids-contempt-charges-for-ai-generated-fake-citations/?slreturn=20250825122622
[4] Josh Taylor, ‘Australian lawyer caught using ChatGPT filed court documents referencing non-existent cases’. The Guardian, 1 February 2025. https://www.theguardian.com/australia-news/2025/feb/01/australian-lawyer-caught-using-chatgpt-filed-court-documents-referencing-non-existent-cases
[5] Beyond Law School, ‘How Should Lawyers Use AI: Lessons from Australia’s Guidelines’, published on 16 June 2025, https://beyondlawschool.in/2025/06/16/how-should-lawyers-use-ai-lessons-from-australias-guidelines/
[6] ‘What are AI hallucinations?’, Google Cloud, https://cloud.google.com/discover/what-are-ai-hallucinations
[7] ‘Google Workspace Additional Terms of Service’. Google Workspace. https://workspace.google.com/terms/service-terms-20250218/
[8] US Copyright Office. Copyright and Artificial Intelligence Part 2: Copyrightability Report (Washington DC: US Copyright Office, 2024). https://www.copyright.gov/ai/Copyright-and-Artificial-Intelligence-Part-2-Copyrightability-Report.pdf
[9] US Copyright Office, Copyright and Artificial Intelligence Part 2: Copyrightability Report (Washington DC: US Copyright Office, 2024), 19-20.
[10] Government of the Netherlands. ‘Contract of employment’. https://business.gov.nl/regulation/contract-employment/
[11] Beyond Law School, ‘Mastering Time Management: Essential Skills for Lawyers’, published 7 January 2025, https://beyondlawschool.in/2025/01/07/mastering-time-management-essential-skills-for-lawyers/
[12] American Bar Association Standing Committee on Ethics and Professional Responsibility, ‘Formal Opinion 512: Generative Artificial Intelligence Tools’ (American Bar Association, July 2024) https://www.americanbar.org/content/dam/aba/administrative/professional_responsibility/ethics-opinions/aba-formal-opinion-512.pdf
[13] American Bar Association, ‘ABA issues first ethics guidance on AI tools’ (ABA News, July 2024) https://www.americanbar.org/news/abanews/aba-news-archives/2024/07/aba-issues-first-ethics-guidance-ai-tools/
[14] Beyond Law School, ‘The Hidden Rules of Networking in Law Nobody Tells You’, published on 21 November 2024, https://beyondlawschool.in/2024/11/21/the-hidden-rules-of-networking-in-law-nobody-tells-you/
Leave a Reply