Written by Rajat Gupta and Navya, students at Himachal Pradesh National Law University, Shimla.
On 17 February 2026, hearing a writ petition that had nothing to do with technology, the Chief Justice of India paused to flag something else. “We are alarmed to reflect… some of the lawyers have started [using] AI to draft. It is absolutely uncalled for,” CJI Surya Kant told the bench. Justice B.V. Nagarathna gave the example that stuck a citation to “Mercy v. Mankind,” a case that exists nowhere except inside a language model, while the Chief Justice noted a “series” of similarly invented judgments in a colleague’s courtroom too. Judicial discretion has always leaned on a human element: memory and scepticism. The question here is what happens to that element once courts start leaning on a tool that can be this confidently wrong: does AI assist judgment, or quietly start standing in for it?
Answering that means first being precise about what “AI in the courtroom” actually covers, because courts are experimenting with three distinct kinds of tool, and the difference matters. The first is retrieval and drafting help SUPACE, the Supreme Court’s own research portal launched in 2021, is built explicitly to extract facts and surface precedent, not to decide. The second is predictive risk-assessment, best known abroad through COMPAS, which scores bail, sentencing and parole in the United States. The third is general-purpose generative AI the ChatGPTs of the world, used informally for summaries and comparative research. What follows uses a specific vocabulary for the risk each creates: automation bias, algorithmic bias, the “black box” problem, and what might be called an algorithmic straitjacket, where a tool built to assist starts to quietly narrow what a judge feels free to decide.
India’s own experiments began small and hedged. In March 2023, Justice Anoop Chitkara of the Punjab and Haryana High Court asked ChatGPT how bail jurisprudence treats cases involving cruelty while deciding Jaswinder Singh v. State of Punjab then recorded in the order itself that the answer had no bearing on the actual decision. The Manipur High Court followed a similar script in May 2024, turning to Google and ChatGPT to research service conditions for a dismissed Village Defence Force volunteer after the state’s own affidavit fell short, and reinstated him on natural-justice grounds. Both orders treated AI as a supplement, carefully disclosed on the record. Three years on, the caution built into that first order looks less like extra caution and more like the only thing standing between “assistive” and “determinative.”
None of this is happening in a vacuum. India’s courts carry a backlog running into the tens of millions, and the appeal of a tool that can triage filings, surface similar precedent, and apply a rule the same way every time is genuine, not manufactured. Daniel Kahneman, Olivier Sibony and Cass Sunstein gave this a name worth borrowing: “noise,” the unwanted variability where two judges or the same judge on different days reach different results on similar facts for no principled reason. A system drawing on prior outcomes promises to shrink that variability and some of the bias intuition alone cannot self-correct. For a system this overworked, that promise is genuinely tempting and, as what follows traces, genuinely incomplete.
The efficiencies described above carry a mirrored cost, and the first is psychological before it is technical. Automation bias is well documented in human-factors research: once a system is perceived as reliable, people stop independently verifying its output and defer to it even when it is wrong.6 Judges are not exempt, and behavioural studies go further, finding that people often prefer algorithmic judgment to their own precisely because it appears more consistent than intuition.
This produces what might be called a cognitive tax. If a risk tool recommends ten years and a judge, weighing the specific facts, believes five is right, accepting the ten needs no explanation; ordering the five demands one a written justification, anticipating an appeal that will ask why the algorithm was overridden. Conformity becomes the path of least resistance, and across enough cases this hardens into a de facto rebuttable presumption favouring the machine, quietly shifting the burden onto the judge who wants to exercise ordinary discretion.
The Wisconsin Supreme Court sensed this danger even while approving the COMPAS risk-assessment tool in State v. Loomis, insisting that any score be paired with written warnings and never serve as “the determinative factor” in sentencing. That such a warning was thought necessary says something: courts already anticipated judges leaning on the number harder than the caution allowed.
Bias risk is not confined to over-reliance; it is compounded by what the tool is trained on. When ProPublica examined COMPAS scores for defendants in Broward County, Florida, it found that Black defendants who did not reoffend were nearly twice as likely as white defendants to be wrongly flagged as high-risk a finding contested on methodology but never persuasively explained away. A model trained on years of policing and sentencing data will reproduce whatever patterns it carries, including discrimination rather than conduct. Feed it India’s own uneven geography of policing, and the same mechanism could quietly harden existing disadvantage into a “risk score” a kind of digital redlining dressed up as neutral computation.
For Indian courts this is a constitutional concern, not only a technical one. Article 14 requires that state action not be arbitrary, and the Supreme Court has long read Article 21’s guarantee of personal liberty to demand a procedure that is fair, just and reasonable, not merely one that exists on paper. A bail or sentencing process that imports unexamined statistical bias fails that test regardless of how efficient it looks on a dashboard.
A third risk sits beneath the first two: many of these tools are, in Frank Pasquale’s phrase, black boxes systems whose outputs are visible while the reasoning behind them is not. A party told that her risk score is “high” cannot meaningfully contest a number she cannot see the working for, and an appellate court reviewing the resulting order meets the same wall. Indian law treats the giving of reasons as inseparable from judicial power: the Supreme Court has called it the soul of justice, a discipline that restrains arbitrariness and lets an order be tested on appeal. An order shaped by an unexplained score sits uneasily with that discipline, however sound the underlying statistics turn out to be.
The stakes go beyond doctrine. Recent experimental research across roughly 1,800 respondents found that people trust human judges more than algorithmic ones and are less willing to bring a dispute before a court that uses one, with the gap widening for cases carrying real emotional weight. Efficiency gained at the console can be legitimacy lost in the courtroom a trade that Indian courts, already working to build public confidence in a system associated with delay, can ill afford to make silently.
Independence is usually understood as freedom from the executive or from litigants. A subtler version is now emerging: freedom from the tool sitting on the judge’s own desk. Where a registry quietly begins to expect that a recommendation will be used, or that departures will be logged and explained, independence is not withdrawn by decree it erodes by increment, through the accumulated weight of paperwork nobody wants to generate.
Even researchers most enthusiastic about prediction in sentencing concede the limit. In their widely cited study of machine-learning models trained on hundreds of thousands of bail decisions, Kleinberg and coauthors note that judges weigh considerations a model never sees remorse, family circumstance, the particular cruelty or particular mercy of a case. That capacity for empathy is precisely the terrain equity has always occupied: correcting a rule for the case its drafters never anticipated. A model trained on averages is, almost by construction, poor at recognising the exception. If sentencing gravitates toward the statistical mean because the mean is what the machine knows how to defend, the law keeps its consistency at the cost of the thing consistency was meant to serve treating each person as more than a data point.
None of this argues for turning courts away from these tools altogether. The scale of India’s case backlog makes the pull toward automation genuinely understandable, and portals built explicitly to assist rather than decide show that the line between aid and authority can be held by design. But a glass gavel is still a gavel, and a transparent instrument does not by itself guarantee a transparent process behind it. What would matter is unglamorous: disclosure whenever a recommendation is consulted, a written record of why it was accepted or set aside, and appellate scrutiny alert to quiet conformity and not only to open error. Judges have always worked with imperfect aids precedent, statistics, expert opinion and absorbed them without surrendering judgment. AI earns a place on that list only if courts insist, at every stage, on remaining the ones doing the deciding.


Leave a Reply