The Prompt | The Precedent and | The Penalty:  Fake Law Citations and Real-World Consequences

“Can machines think?”

In the 1950’s, when Alan Turing posed this question, the age of artificial intelligence was a subject of scientific speculation. In the current paradigm, AI systems can now generate legal arguments, analyse evidence, create original content, and influence decisions that affect human lives. The question is no longer merely whether machines can think, but how the law should respond when their outputs carry real-world consequences. AI can make routine research and drafting more efficient, allowing legal professionals to devote greater attention to legal reasoning and strategy. However, its usefulness depends on the reliability of its outputs, particularly when legal arguments and judicial decisions rely on accuracy. 

The consequences of this reliance become particularly acute when AI-generated inaccuracies enter the judicial process. This concern came into focus on 2 July 2026, when the Supreme Court of India delivered a judgment that examined the use of AI in legal proceedings. In Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd.,¹ The Supreme Court declared that a decision based on fabricated legal material is ‘no decision in the eyes of the law,² and set aside the NCLT and NCLAT orders. By holding that a decision tainted by fake material is a nullity, the Court is exercising its authority to correct its own records and to maintain the integrity of the judicial process. This also brings into focus the Supreme Court’s constitutional status as a Court of Record under Article 129 of the Constitution, which reflects its role in preserving the authority and integrity of the judicial process. The Court’s warning that such material is “invisible, insidious, and catastrophic”³ reflects a recognition that fake citations do not merely mislead the parties; they corrupt the institutional cadence of the court itself. 

The Supreme Court directed the Bar Council of India to constitute a committee to address the issue of advocates submitting fake or hallucinated material as precedent. The Court’s language, ‘constitute a committee’ and ‘prescribe guiding principle’,⁴ points towards the latter, but the BCI must decide which regulatory mechanism it will adopt. It may frame a set of formal rules, guided by the Advocates Act, 1961,⁵ that establish legally binding obligations and guidelines, offering greater flexibility in responding to the evolution of AI in legal practice. The Pooja Ramesh judgment leaves the precise regulatory approach to be determined by the BCI. Alongside this, it raises questions about how responsibility for AI-generated legal work should be addressed within the Indian legal profession.

If AI can conduct legal research and prepare documents, what responsibilities does a lawyer have?

Case law may be artificially generated, the interpretation may be wrong, and the right answer may not align with a client’s specific interests. The question is not only whether AI should be used in legal practice, but also how its use can align with professional expertise and responsibilities while maintaining information confidentiality and the integrity of the court proceedings.

AI in Legal Research: Efficiency, Accuracy and Judicial Accountability

Generative AI, a subset of artificial intelligence, creates content such as text, images and videos by learning patterns from existing training data. In legal practice, it can assist with summarising judgments, identifying precedents, translating texts and preparing drafts. AI makes this task faster, but it does not eliminate the need for quality control and supervision. However, its outputs may contain fictitious cases, inaccurate citations or erroneous interpretations of law which is  commonly referred to as AI hallucinations. The Supreme Court of India’s White Paper on AI and Judiciary,⁶ issued in November 2025, examines the use of artificial intelligence in judicial administration and legal work, including tools such as the Supreme Court Portal for Assistance in Court’s Efficiency (SUPACE), Supreme Court Vidhik Anuvaad Software (SUVAS), Technology Enabled RESolution (TERES), and Legal Research Analysis Assistant (LegRAA). The White Paper recognises that AI can assist with legal research, translation, transcription and case management, but also identifies risks arising from inaccurate outputs, fabricated legal authorities, algorithmic bias and breaches of confidentiality. It therefore recommends human oversight, systematic verification of AI-generated material, institutional safeguards and clear guidelines governing the responsible use of AI in judicial institutions.

The use of AI technologies in research, transcription, and translation may increase the efficiency of obtaining legal information in Indian courts, where there are backlogs. However, the use of these tools does not absolve lawyers of their responsibility to assess the law, verify cited authorities and determine their applicability to the client’s case. The consequences of failing to exercise this scrutiny are evident in judicial decisions involving fabricated legal references. In Greenopolis Welfare Association v. Narender Singh,⁷ The Delhi High Court adjudicated upon a petition containing references to non-existent precedents and passages that could not be traced to the judgments cited. Similar concerns arose before the Supreme Court in Gummadi Usha Rani v. Sure Mallikarjuna Rao,⁸ where the Court took cognisance and held that relying on AI-generated fake precedents in a judicial order constitutes “misconduct” rather than a mere adjudicatory error. In Vijay Ghanshyam Gadiya v. Union of India,⁹ The Supreme Court reportedly set aside a customs penalty of approximately ₹425 crore, holding that it was imposed in reliance on non-existent precedents. In Deepak Raheja v. Omkara Assets Reconstruction Private Ltd.,¹⁰ The Supreme Court addressed an objection that the appellant’s compilation contained non-existent judgments generated using AI. The Court recorded the responses of the Advocate-on-Record and senior counsel and considered whether exemplary costs should be imposed. 

Nevertheless, the incident raises questions about a lawyer’s responsibility to keep a check on the legal submissions made before a court. Together, these cases highlight the need to verify legal authorities, whether they originate from lawyers, tribunals or administrative authorities. It eventually leaves us with a twofold regulatory question :

What safeguards should govern AI use in legal proceedings, and how should responsibility be allocated when those safeguards fail?

The Legal Framework: Regulation, Confidentiality and Liability 

India’s approach to AI governance in the legal sector is developing through judicial decisions, institutional policies and proposed regulations. The Kerala High Court’s policy¹¹ on 19 July 2025 restricts the use of AI tools in the district judiciary. Among other safeguards, it prohibits AI from determining findings, reliefs, orders or judgments, restricts cloud-based tools to approved services and provides for verification and auditing. In November 2025, the Supreme Court’s White Paper outlined principles including human oversight, accuracy, confidentiality, fairness, bias prevention and specialised verification of translated material. On 3 June 2026, the Supreme Court’s AI Committee published the preliminary Draft Regulations for Use of Artificial Intelligence in Courts, 2026.¹² 

The existing constitutional and statutory provisions already address several related concerns, including equality, privacy, professional conduct and access to legal representation. The use of AI in legal practice must be assessed against constitutional guarantees, professional obligations and evidentiary rules. Under the Advocates Act, 1961, Sections 35,¹³ 7¹⁴ and 49¹⁵ provide the framework for professional discipline, standards and rule-making. The primary sources of a lawyer’s duty of skill and diligence are the law of professional negligence and the disciplinary framework under the Advocates Act, 1961. The Bharatiya Sakshya Adhiniyam, 2023, addresses the admissibility of electronic records under Sections 61¹⁶ and 63,¹⁷ while Section 132¹⁸ protects professional communications. Together, these provisions raise questions about accuracy, confidentiality, accountability and the evidentiary status of AI-generated material.

Lawyers using third-party AI platforms must consider the risks of uploading pleadings, contracts, correspondence, and evidence that contain personal or confidential information. The right to privacy recognised in K.S. Puttaswamy v. Union of India¹⁹ also makes the protection of confidential client information relevant to AI adoption. The Digital Personal Data Protection Act, 2023,²⁰ and the DPDP Rules, 2025,²¹ provide a framework for processing personal data. Section 8 of the Act²² retains the Data Fiduciary’s responsibility for compliance when a Data Processor undertakes processing and addresses contractual arrangements and security safeguards. Law firms should therefore assess AI providers, data storage and processing practices, model-training policies, retention periods, deletion mechanisms and breach-reporting procedures before adopting their tools. 

Responsibility extends to supervising lawyers who fail to check work prepared by juniors or produced through automated tools. In a large firm, an AI tool may be selected by an innovation committee, configured by IT, used by a junior associate, and reviewed by a partner. When a fabricated authority is presented to a court, the question of responsibility cannot be answered solely by identifying the advocate who signed the filing. The US decision in Mata v. Avianca, Inc.,²³ illustrates the consequences of filing fictitious authorities without proper verification. Thus, an optimal framework must allocate responsibility across the production chain among the lawyer who selected the tool, the junior who operated it, the partner who reviewed the output, and the firm that procured it. 

The [DRAFT] REGULATIONS FOR USE OF ARTIFICIAL INTELLIGENCE IN COURTS, 2026, by contrast, place responsibility on ‘the person submitting’²⁴ the material. Preparing the profession requires a practical framework built around five measures: firstly, verifying every legal authority against an official source or reliable law report; secondly, adopting approved-tool policies and reviewing vendor contracts; thirdly, retaining human control over legal analysis and litigation strategy; fourthly, providing training on hallucinations, bias, privacy, intellectual property and evidentiary authentication; and finally, making reliable AI tools and training accessible to smaller firms and individual practitioners. Law schools can also teach responsible use by requiring students to compare AI-assisted research with findings drawn from primary sources.

These safeguards need not restrict the profession’s use of AI. Instead, they can prepare lawyers for emerging work in AI governance, data protection, intellectual property and the authentication of digital evidence, while preserving their existing duties to clients and courts. The challenge is to adopt useful technology without compromising professional standards.

Conclusion

The question is not whether the Indian legal profession is ready to use artificial intelligence, it is about being responsible in its usage in the profession. The cases discussed in the article reveal that incidents of incorrectly generated sources are not merely technical glitches. The moment such sources enter the judicial process, they may compromise the reliability of judicial decision-making and require time to correct mistakes. However, taking verification for granted alone will not solve the problems. It is important to note that independent judgment is an essential part of the lawyer’s profession. Still, individual responsibility cannot be used to address issues concerning the use of AI. 

A defensible framework must acknowledge the BCI’s institutional limits. The BCI’s disciplinary machinery is already overburdened, and its rule-making under Section 49 of the Advocates Act, 1961,  requires Chief Justice of India approval. Placing the primary responsibility for standard-setting and enforcement solely on the BCI is therefore unrealistic. A distributed accountability model is preferable in which the BCI sets minimum standards of verification and supervision, calibrated to the size and resources of the practice; then High Courts exercise supervisory jurisdiction over filings that cite unverified authorities, firms implement internal controls proportionate to their size, including tool approval, audit trails, and partner review; and individual lawyers retain ultimate responsibility for what they sign. This model does not absolve the BCI; it recognises that standard-setting and enforcement need not be concentrated in a single body. The BCI’s existing guidance on AI is, at present, non-existent. The committee constituted pursuant to Pooja Ramesh Singh is therefore operating on a blank slate, which makes the choice of instrument and the allocation of enforcement responsibility all the more consequential. 

India, hence, requires a perspective that merges professional accountability with institutional responsibility. The mentioned principles should be practical enough for individual lawyers and small offices to observe, rather than compliance norms that are affordable only for corporations. The readiness of the legal community should be gauged not just by the introduction of novelties and the dissemination of regulations. It should be evaluated by the responses to questions about how AI-involved practices can be checked, whether errors can be detected, how confidentiality can be ensured, and how responsibility can be assigned in the event of an error. Although AI can influence the provision of legal services, it should not lower the standards of their evaluation.

Citations

1. Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. & Anr., 2026 INSC 668, Civil Appeal No. 11950 of 2025. ↩︎
2. Id. at para. 7. ↩︎
3. Id. at para. 6. ↩︎
4. Id. at para. 9. ↩︎
5. The Advocates Act, 1961 (Act 25 of 1961). ↩︎
6. Centre for Research and Planning, Supreme Court of India, White Paper on AI and Judiciary (November 2025). ↩︎
7. Greenopolis Welfare Association v. Narender Singh, CM(M) No. 1909 of 2025, CM APPL. Nos. 61372 of 2025 and 61371 of 2025 (Delhi High Court, New Delhi). ↩︎
8. Gummadi Usha Rani v. Sure Mallikarjuna Rao & Anr., SLP (C) No. 7575 of 2026, para. 7 (Supreme Court of India, order dated 27 February 2026). ↩︎
9. Vijay Ghanshyam Gadiya v. Union of India & Anr., Civil Appeal arising out of SLP (C) No. 15605 of 2026, para. 2 (Supreme Court of India, 2026). ↩︎
10. Deepak Raheja v. Omkara Assets Reconstruction Private Ltd., 2026 INSC 1134, para. 74 (Supreme Court of India, judgment dated 15 October 2026). ↩︎
11. Kerala High Court, Guidelines on the Use of Artificial Intelligence (AI) Tools in the District Judiciary (19 July 2025), ↩︎
12. Draft Regulations for Use of Artificial Intelligence in Courts, 2026. ↩︎
13. The Advocates Act, 1961 (Act 25 of 1961), s. 35. ↩︎
14. The Advocates Act, 1961 (Act 25 of 1961), s. 7. ↩︎
15. The Advocates Act, 1961 (Act 25 of 1961), s. 49. ↩︎
16. The Bharatiya Sakshya Adhiniyam, 2023 (Act 47 of 2023), s. 61. ↩︎
17. The Bharatiya Sakshya Adhiniyam, 2023 (Act 47 of 2023), s. 63. ↩︎
18. The Bharatiya Sakshya Adhiniyam, 2023 (Act 47 of 2023), s. 132. ↩︎
19. K.S. Puttaswamy v. Union of India, (2017) 10 SCC 1. ↩︎
20. The Digital Personal Data Protection Act, 2023 (Act 22 of 2023). ↩︎
21. Digital Personal Data Protection Rules, 2025. ↩︎
22. The Digital Personal Data Protection Act, 2023 (Act 22 of 2023), s. 8. ↩︎
23. Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023) ↩︎
24. Supreme Court of India, [Draft] Regulations for Use of Artificial Intelligence (AI) in Courts, 2026, reg. 43(6). ↩︎

Expositor(s): Adv. Stephin Sinu Oommen