Brazilian court AI catches hidden prompt injection

A Brazilian labour court’s AI assistant flagged a trick its human readers could not see: an instruction typed in white text on a white background inside a legal filing. According to EL PAÍS, the hidden line told the system to answer the case superficially and avoid challenging the documents, and the judge sanctioned the lawyers who submitted it.

The episode turns a familiar security problem into an institutional one. Courts are adopting AI tools to sort, classify and summarise filings because modern cases arrive as sprawling PDFs and attachments, and judges and clerks are time-poor. EL PAÍS reports that the Brazilian system, called Galileu, proposes draft rulings and analyses documents as an assistant, while formal legal analysis and evidence evaluation remain mandatory for humans. That division of labour still creates a narrow choke point: if the machine can be nudged to misread or ignore parts of the record, the human reviewer may never be shown what matters.

The technique is known as prompt injection—embedding instructions aimed at the model rather than the judge. In the Brazilian case, the instruction was designed to be invisible to the eye but legible to the software, effectively turning the court’s automation into an uninvited co-counsel. Marcelo Quaglia, an Argentine judge and professor cited by EL PAÍS, described this as an expected development rather than science fiction, noting that any workflow that routes authority through a text-processing system invites adversarial inputs.

Spain is moving in the same direction, but with different guardrails. EL PAÍS quotes lawyer Abel Gende saying Spain’s General Council of the Judiciary issued an order in January regulating—rather than banning—AI use by judges, explicitly covering analysis, classification and structuring of case documents. The Brazilian incident illustrates why “assistive” can be a legal fiction: once a tool shapes what the judge sees first, it shapes what the judge has time to see at all.

Brazilian courts have reportedly identified similar attempts elsewhere, including a case in São Paulo in which a prompt asked the system to grant legal aid, approve urgent measures and summon the defendant while asserting that all documentation had been submitted. Galileu had been in use for about a year when the first attempt was detected.

The sanctioned lawyers could not be reached by EL PAÍS through their LinkedIn profiles. The hidden text, meanwhile, was reachable by the court’s machine on the first read.