AI hallucinated citations: the risk for Indian lawyers

TL;DR: General-purpose AI tools invent case citations that look real but do not exist. A US court sanctioned two lawyers in 2023 after they filed a brief containing six fabricated AI-generated citations. Indian courts are watching. No Bar Council regulation covers AI yet, but professional conduct rules still apply, and the stakes - sanctions, struck pleadings, reputational damage - are the same. This piece explains why hallucinations happen, what a fake citation looks like, what the global sanction trend means for Indian practitioners, and how retrieval-grounded legal AI tools that check citations against real judgments reduce the structural risk.

On this page

What is an AI hallucinated citation?

The term “hallucination” in AI refers to outputs that are confidently stated but factually wrong. In the legal context, an AI hallucinated citation is a case reference - complete with party names, court, year, neutral citation, and sometimes page numbers - that does not correspond to any real judgment.

The danger is not that the output looks obviously wrong. It looks exactly right. The citation follows the correct format for the jurisdiction. The party names are plausible. The holding described matches the legal point being argued. The problem is that when you go to verify it, the case does not exist.

For a lawyer, this is not a minor data error. It is a representation to a court that a judgment exists and stands for a proposition of law. If the judgment is fabricated, that representation is false. Courts take false representations very seriously, and rightly so.

Understanding why this happens requires a short detour into how large language models actually work - because the mechanism is not random error. It is a predictable consequence of how these systems are designed.

Why language models fabricate case law

A large language model is a statistical text prediction engine. It is trained on enormous quantities of text and learns to predict, at each step, which token (roughly, which word or word-fragment) is most likely to follow the tokens that came before. The model does not look anything up. It does not query a database. It does not retrieve a document. It generates text.

This distinction matters enormously. When you ask a general-purpose LLM to cite cases on, say, the scope of judicial review under Article 226, it does not search a legal database. Instead, it generates text that looks like what a legal brief citing Article 226 cases would look like. If the model has seen enough legal text during training, it can produce output that is fluent, properly formatted, and legally coherent - while citing cases that were never decided.

The model is not “lying” in any intentional sense. It has no access to ground truth. It generates plausible text, and plausible legal text contains citations in a particular format. The model fills in that format with names, numbers, and citations that match the pattern of real citations it encountered during training. Some of those generated citations will happen to correspond to real cases. Others will not. The model cannot tell the difference, and it will not tell you which is which.

This is not a bug that will be patched in the next version. It is an architectural feature of how generative AI works. The only structural fix is to ground the model’s outputs in a verified set of real documents - so that citations are drawn from actual judgments, not generated from training data patterns.

What a fake citation looks like - and why it is so convincing

If hallucinated citations looked obviously wrong, the problem would be self-correcting. Lawyers would notice immediately. The unsettling reality is that fabricated citations are often indistinguishable from real ones until you go to find the original judgment.

A typical hallucinated Indian citation might look like this:

Rajendra Kumar v. State of Haryana, (2019) 7 SCC 412

Everything about this is plausible. “Rajendra Kumar” is a common Indian name. “State of Haryana” is a real respondent in thousands of cases. The SCC reporter format is correct. The year and volume are reasonable. The page number follows the correct pattern. If you are reading this in a brief and you are not in a position to verify it immediately, you might well accept it.

The model generates the case name by combining common elements from Indian case law it was trained on. It generates the reporter citation by following the SCC format it has seen repeatedly. It generates the holding by producing legal text that matches the argument being made. The result is a citation that passes a superficial plausibility test but fails the only test that matters: does this case actually exist?

Consider what a plausible fake looks like compared to a real one:

FeatureReal citationHallucinated citation
Party namesTraceable in court recordsPlausible but unverifiable
Reporter/neutral citationMatches actual reporterFollows format; volume/page may not exist
YearCorresponds to real decisionPlausible year, may not match any filing
Holding describedMatches actual ratioConsistent with argument; may invert real holding
Findable on SCC Online / Indian KanoonYesNo - or a different case at that citation
Findable via court order searchYesNo - or case with different parties
Cross-references consistentYesTypically inconsistent on deeper check

The last two rows are where verification catches the problem. But verification takes time, and under deadline pressure, it is easy to skip.

Mata v. Avianca: the case that changed the conversation

In June 2023, a US District Court in New York sanctioned two attorneys after they submitted a brief in a personal injury case - Mata v. Avianca, Inc. - that cited six cases generated by ChatGPT. None of those cases existed. When opposing counsel and then the court asked for copies of the cited cases, the attorneys could not produce them because there were no cases to produce.

The court’s opinion was scathing. The attorneys had used an AI tool without understanding that it could fabricate citations, had not independently verified any of the cases, and had submitted the brief without disclosing the use of AI. When the problem was raised, they initially doubled down rather than investigating. The court imposed sanctions, required completion of continuing legal education on AI, and the case became a widely-cited cautionary example worldwide.

Several aspects of Mata are worth examining carefully:

The attorneys were not negligent in an obvious way. They were using a tool they believed was reliable. The AI produced output that looked professional and well-reasoned. The citations appeared in a format lawyers recognise. The error was not in failing to check for obviously wrong output - it was in not understanding that a general-purpose AI tool is architecturally incapable of guaranteeing citation accuracy.

The cover-up made things worse. When the court first raised questions, the attorneys sought to verify the citations by asking ChatGPT to confirm they were real. ChatGPT confirmed they were real. This is precisely what you would expect: a hallucinating model will also hallucinate confirmation of its hallucinations. The attorneys did not understand this.

The sanctions were not about AI per se. The court’s primary concern was the duty of candour to the tribunal and the duty of competence. Both are jurisdiction-independent professional obligations. The AI tool was incidental; the professional failure was submitting unverified citations to a court.

This framing matters for Indian lawyers. The Mata sanctions are a US story, but the underlying professional obligations - verify what you file, do not mislead the court - exist in every jurisdiction, including India.

The global sanction trend since Mata

Mata was not an isolated incident. Since 2023, similar AI citation fabrication problems have surfaced in courts across multiple jurisdictions - in the UK, Australia, Canada, and elsewhere. In each instance, the core failure was the same: a lawyer or litigant used a general-purpose AI tool, accepted its output without independent verification, and submitted fabricated citations to a court.

The pattern across these incidents tells us several things:

General-purpose AI tools are not improving fast enough to make this problem go away. Even as models become more capable, the fundamental architecture - text prediction rather than retrieval - means that citation fabrication remains a live risk. Some tools have added retrieval components, but this varies by product and is not always transparent to the user.

Courts are becoming less tolerant, not more. Early incidents were sometimes treated with a degree of leniency on the grounds that lawyers were unfamiliar with AI risks. As awareness increases, courts are more likely to treat failure to verify AI output as a culpable professional failure rather than a forgivable mistake.

The professional conduct framework is catching up. Bar associations and law societies in several jurisdictions have issued guidance - or are developing it - on the use of AI in legal practice. Some require disclosure of AI use. Some require that any AI-generated content be independently verified. The direction of travel is clear.

For Indian lawyers monitoring this space, the trend line matters. What is currently an emerging issue in other jurisdictions may become a live enforcement issue in India as AI adoption in legal practice accelerates.

The Indian context: where things stand in mid-2026

As of mid-2026, the Bar Council of India has not issued specific regulations governing the use of AI in legal practice. There is no mandatory disclosure requirement for AI-generated legal work product. There is no Indian sanction case directly analogous to Mata v. Avianca that has been reported and widely publicised.

This does not mean Indian lawyers face no risk. It means the regulatory framework has not yet caught up with practice.

Several dynamics are worth tracking:

Indian court digitisation is making verification easier - and therefore verification failures harder to excuse. With eCourts, the Supreme Court’s judgment portal, High Court websites, and platforms like Indian Kanoon providing searchable access to judgments, the argument that verification was impractical grows weaker by the year. If a citation cannot be found on these platforms, that is a significant red flag.

Senior counsel and junior advocates have different risk profiles. A junior advocate preparing a first draft may use AI tools without fully understanding their limitations. A senior advocate signing off on a petition relies on that draft. Both bear professional responsibility for what is filed. As AI becomes more common in chambers, the question of who verified what becomes more complex.

Indian courts are observing global developments. Judges and court administrators are aware of Mata and similar cases. It would be reasonable to expect that Indian courts - particularly the Supreme Court and High Courts with international legal exposure - will look with increasing seriousness at any citation that cannot be produced on request.

The general-purpose AI tools most commonly used are not built for Indian legal research. ChatGPT, Gemini, and similar tools are trained on broad internet text. Their knowledge of Indian case law is limited and uneven. The risk of hallucination is higher, not lower, for Indian law than for US or UK law where more structured legal text was available in training data.

Professional conduct obligations that already apply

Even without specific AI regulation, Indian lawyers operate under professional conduct rules that are directly relevant to citation accuracy.

The Advocates Act, 1961 and the Bar Council of India Rules impose duties that include acting with honesty and integrity, not misleading the court, and maintaining the standards of the profession. These are not aspirational - they are enforceable obligations.

When a lawyer files a written submission citing a case, they are making a representation to the court. That representation is that the cited case exists, says what it is said to say, and is good law. If any of those elements is false - because the case was fabricated by an AI tool - the representation is false. That is a professional conduct issue regardless of whether the lawyer knew the citation was fabricated.

The standard is not subjective good faith. The question courts ask is whether the lawyer took reasonable steps to verify what they filed. In the current environment, where the risk of AI hallucinations is publicly known and the tools to verify Indian citations are widely available, “I relied on the AI” is unlikely to be a sufficient answer.

The lawyer’s duty to verify AI output exists independently of any specific AI regulation, and it applies to every citation in every filing.

How Indian court practice amplifies the risk

Certain features of Indian legal practice make citation hallucinations particularly dangerous.

Volume of filings. Indian courts handle enormous volumes of litigation. Junior advocates and law clerks are under significant time pressure. The temptation to use AI to generate research quickly, and to skip verification under deadline pressure, is real.

Citation reliance in arguments. Indian legal culture places heavy weight on cited precedent. Submissions regularly cite multiple cases. A brief citing fifteen cases has fifteen opportunities for a hallucination to slip through. If one case in fifteen is fabricated and is specifically relied upon in argument, the consequences can be severe.

Difficulty of cross-jurisdictional citations. When Indian lawyers cite cases from other jurisdictions - UK Privy Council, English courts, US decisions on analogous questions - the verification challenge is greater. These citations are harder to check quickly, and general-purpose AI tools trained primarily on US and UK text may produce more plausible-sounding but equally fabricated foreign citations.

The gap between drafting and filing. In many chambers, the advocate who drafts is not the advocate who appears. Verification responsibility can fall through the gap between the two. Clear internal protocols about who verifies AI-generated research before it becomes part of a filed document are essential.

See also: how to vet legal AI citation accuracy for a detailed look at verification methods.

The verify-before-you-file checklist

Every Indian lawyer using AI tools for legal research should have a verification protocol. Here is a practical checklist:

For every citation generated by any AI tool:

  • Search the case name on Indian Kanoon or the Supreme Court / High Court official portal
  • Confirm the neutral citation or reporter citation matches the judgment found
  • Read at least the headnote - confirm the holding matches what the AI described
  • Check that the ratio cited is actually in the judgment, not just a plausible reading
  • Run the citation through a citator or good law check to confirm it has not been overruled
  • If citing for a specific proposition, locate the specific paragraph in the judgment

For foreign citations:

  • Search on the jurisdiction’s official case law database (BAILII for UK, Austlii for Australia, etc.)
  • Do not accept the AI’s description of a foreign holding without reading the actual case
  • Be especially cautious with US circuit court citations - the model may generate plausible but nonexistent case names in the correct format

Before filing:

  • Every citation in the final draft has been independently verified by a human who read the judgment
  • No citation appears that could only have come from AI output without subsequent verification
  • If AI was used in drafting, the supervising advocate is satisfied with the verification process

Red flags during verification:

  • Citation not findable on any authoritative database
  • AI-generated copy of the judgment that differs from the version on official databases
  • Holding described by AI that is significantly more favourable than what the actual case says
  • Case name, citation format, or court all correct but year or volume inconsistent with actual publication

This checklist is not exhaustive. Chambers handling sensitive matters - criminal cases, constitutional matters, large commercial disputes - should consider more rigorous verification protocols. The AI legal research guide for India covers these in greater depth.

What to do if you discover a fake citation

Suppose you discover that a citation in a draft - or worse, in a document already filed - does not exist. What should you do?

If the document has not yet been filed:

Remove the citation immediately. If the legal point it was meant to support is still valid, find a real case that supports it. Do not substitute another AI-generated citation without verification. If you cannot find a real case, do not make the argument that depends on it.

Review all other citations in the document using the checklist above. A single hallucination in a draft suggests the research process was not adequately verified; there may be others.

Consider whether your AI research workflow needs to change before you use it again.

If the document has already been filed:

This is a more serious situation. The appropriate course depends on the stage of proceedings and the court, but some principles apply across situations.

Do not wait. The longer a false citation sits in a filed document, the worse the professional conduct picture looks. Courts treat prompt disclosure far more favourably than continued reliance on a false citation.

Seek guidance from a senior colleague or bar association ethics resource before taking action. The right procedural step - corrigendum, application to amend, direct disclosure to the court - depends on the specific situation.

Do not ask the AI tool to verify its own citation. As Mata demonstrated, the model will typically confirm its own hallucination. Verification must be independent.

Document what happened and when you discovered it. If professional conduct questions arise later, a clear record of how and when the problem was identified and what you did about it will matter.

The distinction that matters most when evaluating legal AI tools is not how sophisticated the model is. It is whether the system generates citations or retrieves them.

A general-purpose LLM generates citations. It produces text that looks like a citation based on patterns in its training data. It has no mechanism for checking whether the citation corresponds to a real judgment. That is not a feature gap; it is an architectural characteristic.

A retrieval-grounded legal AI tool works differently. When you ask a question, it searches a verified corpus of real judgments - documents that have been collected, indexed, and stored. The citations in its answers are drawn from those real documents. The model still does language tasks (summarising, explaining, connecting the citation to your question), but the citations themselves come from the database, not from the model’s imagination.

This matters because it changes the error mode. A retrieval-grounded tool might retrieve a case that is less directly on point than you hoped. It might summarise a holding imperfectly. But it will not cite a case that does not exist in its corpus, because the citation comes from the corpus itself.

The important caveat is that retrieval-grounded tools are only as good as the corpus they search. A tool with a small, outdated, or inaccurate database of Indian judgments provides weaker protection than one with comprehensive, verified coverage. This is why the quality and scope of the underlying judgment database matters as much as the AI layer on top.

The comparison between native legal AI and general chatbots goes into more detail on this architectural distinction.

Niyam’s approach: grounded answers, built-in citator

Niyam is a legal AI built specifically for India. Its research tool is retrieval-grounded over more than 72,000 Indian judgments - Supreme Court and High Court decisions that form the backbone of Indian case law. Every answer Niyam provides is cited to a real judgment in that corpus. The citation is drawn from the database, not generated by the language model.

Niyam also includes a built-in citator that lets you check whether a case is still good law - whether it has been distinguished, overruled, or followed in subsequent decisions. This matters because a citation can be real and still be dangerous: a case that has been overruled provides no legal support and may actively mislead.

The honest position is this: Niyam significantly reduces the risk of citation hallucination by design. But it does not eliminate the lawyer’s verification duty. No tool does. The professional obligation to verify what you file is yours. Niyam makes that verification faster and more reliable - it gives you real citations to check, not fabrications to chase - but the checking is still your job.

What Niyam does not do is allow you to skip verification. The purpose of retrieval grounding and a built-in citator is to make verification fast and reliable, not to make it optional.

You can see how Niyam compares to general-purpose AI tools for legal research.

For specific workflows, the Niyam research solution covers case law search and citation, while the citator handles good law checking.

Frequently asked questions

What exactly is an AI hallucinated citation?

An AI hallucinated citation is a case reference produced by a language model that appears real - correct format, plausible party names, proper reporter citation - but does not correspond to any actual judgment. The model generates it from patterns in its training data rather than retrieving it from a verified database of real cases.

Why do large language models hallucinate citations rather than just saying they do not know?

Language models are trained to produce fluent, helpful text. When asked for cases on a legal topic, the model generates text that looks like what an answer citing real cases would look like. The model has no mechanism for distinguishing between “this case exists” and “this text pattern fits what a case citation looks like here.” It produces a confident-sounding citation because that is what its training optimised it to do.

Has any Indian court sanctioned a lawyer for using AI-generated fake citations?

As of mid-2026, no Indian sanction case directly analogous to Mata v. Avianca has been publicly reported. However, the absence of a reported case does not mean there is no risk. Indian professional conduct rules already require lawyers not to mislead courts, and these apply to false citations regardless of how they were generated.

What happened in Mata v. Avianca?

In 2023, a US District Court in New York sanctioned two attorneys who submitted a brief citing six cases generated by ChatGPT, none of which existed. The court found violations of professional conduct rules relating to candour to the tribunal and the duty of competence. The attorneys had not verified the citations and had initially relied on ChatGPT itself to confirm the cases were real.

Does the Bar Council of India have any specific AI regulation?

As of mid-2026, the Bar Council of India has not issued specific regulations governing the use of AI in legal practice. There is no mandatory AI disclosure requirement. Existing professional conduct rules under the Advocates Act and Bar Council of India Rules apply, including duties of honesty and not misleading courts.

Is it a professional misconduct issue even if I did not know the citation was fake?

The professional conduct standard is not purely subjective. The question is whether you took reasonable steps to verify what you filed. In the current environment, where AI hallucinations are publicly known and verification tools are available, relying on AI output without independent verification is likely to be viewed as a failure to take reasonable steps, regardless of subjective good faith.

How can I tell if a citation is real or hallucinated?

The reliable method is independent verification: search the case name and citation on Indian Kanoon, the Supreme Court portal, or the relevant High Court website. Find the actual judgment. Read it. Confirm the holding matches what the AI described. A citation that cannot be found on any authoritative database should be treated as suspect.

ChatGPT and similar general-purpose models generate citations from training data patterns. A retrieval-grounded tool searches a verified corpus of real judgments and draws citations from those actual documents. The error modes are different: retrieval-grounded tools may retrieve less relevant cases, but they will not cite cases that are not in their verified corpus.

No. The professional obligation to verify what you file rests with you. A tool that provides real citations makes verification easier and more reliable, but it does not substitute for the lawyer’s own judgment and checking. You remain responsible for every citation in every document you file.

What should I do if I find a fake citation in a document I have already filed?

Do not wait and do not ask the AI to verify its own citation. Seek guidance from a senior colleague or bar association ethics resource. The right procedural step - corrigendum, application to amend, direct disclosure - depends on the specific situation. Courts treat prompt, honest disclosure far more favourably than delay or continued reliance on a false citation.

Are some types of Indian cases more at risk of hallucination than others?

Cases that are relatively rare in AI training data - older judgments, High Court decisions from smaller benches, specialised tribunal decisions, cases in regional languages - are more likely to be hallucinated because the model has less real data to draw on and more gap-filling to do. Foreign case citations in Indian submissions are also higher risk.

General-purpose tools can help with legal drafting, summarising principles, or exploring arguments. The critical discipline is: never use AI-generated citations without independent verification through an authoritative database. If you cannot verify a citation, do not use it. Many lawyers find that the time spent verifying citations from general-purpose tools exceeds the time saved in drafting.

What is the risk specific to junior advocates and law clerks?

Junior advocates typically prepare drafts that senior advocates review and sign. If a junior advocate uses AI tools without understanding the hallucination risk and does not flag this, the senior advocate reviewing the draft may not know to check AI-generated citations more carefully. Clear chamber protocols about verifying AI output are essential.

A citator lets you check whether a case is still good law - whether it has been followed, distinguished, or overruled by subsequent decisions. This is a separate check from whether the citation is real. A real citation that has been overruled is not useful precedent. Running citations through a citator catches both fabricated citations (which will not appear in the citator’s database) and overruled ones.

What verification should I do for foreign citations in Indian submissions?

For UK cases, use BAILII. For Australian cases, AustLII. For US cases, Google Scholar or Justia provide free access to many decisions. Read the actual case, not the AI’s summary of it. Be especially cautious about US circuit court decisions where the model may generate a plausible case name in the correct format for a case that does not exist.

Does Niyam guarantee that none of its citations will be hallucinated?

Niyam is retrieval-grounded over 72,000+ Indian judgments, meaning citations in its answers come from that real corpus rather than being generated by the model. This structurally addresses the hallucination problem for cases within that corpus. Lawyers should still verify that the cited judgment says what it is said to say, and should check whether it remains good law using the built-in citator.

How is Niyam’s coverage of Indian judgments different from what a general AI tool knows?

General-purpose AI tools are trained on broad internet text. Their coverage of Indian case law is variable, often thin for older judgments and High Court decisions, and not systematically verified. Niyam’s corpus is specifically built for Indian law and covers 72,000+ Supreme Court and High Court judgments, making it far more reliable for Indian legal research than a general-purpose tool.

Is there a way to test whether an AI tool is retrieval-grounded or just generating text?

Ask it for a case citation on a narrow point. Then try to find that case on Indian Kanoon or the Supreme Court portal. If it exists and says what the tool said, that is a positive signal. If it does not exist, or the holding is different, the tool is likely generating rather than retrieving. You can also ask the tool to show you the source document - retrieval-grounded tools can point you to the actual judgment; generating tools cannot.

What practical steps should a law firm take right now?

Adopt a clear policy: no AI-generated citation goes into a filed document without independent human verification. Brief all advocates and clerks on the Mata v. Avianca case and what it means. Consider switching research workflows to tools that are specifically built for Indian law and are retrieval-grounded. Update supervision protocols so there is explicit accountability for who verifies what in every filing.

Start with Niyam’s guides on how to vet legal AI citation accuracy, good law checking, and AI legal research in India. The comparison page gives a direct look at how general-purpose AI tools differ from tools built for Indian legal research.

Start researching with confidence

The hallucination problem is structural, not incidental. General-purpose AI tools were not built to guarantee citation accuracy because their architecture does not permit it. For Indian lawyers, the professional stakes of a fabricated citation are high - and rising as courts and bar associations pay closer attention to AI use in practice.

The right response is not to avoid AI in legal research. It is to use tools that are designed to handle citation accuracy correctly, and to maintain rigorous verification habits regardless of which tools you use. Retrieval-grounded research tools, built-in citators, and a disciplined verify-before-you-file protocol together address the risk that a fake citation ends up in your client’s matter.

Niyam is built for exactly this workflow. Research grounded in real Indian judgments. A citator to check good law. Drafting that works from verified sources. And a team reachable at [email protected] when you have questions.

When you are ready to try it: Start for ₹100 - 200 credits to start, cancel anytime. Questions: [email protected].