A recent federal court opinion brought to light a concerning practice by immigration agents: the use of artificial intelligence, specifically ChatGPT, to generate use-of-force reports. U.S. District Judge Sara Ellis, in a two-sentence footnote within a voluminous 223-page opinion, explicitly raised alarms that this practice could lead to significant inaccuracies and further erode public confidence in law enforcement's handling of immigration crackdowns and related protests in the Chicago area.

Judge Ellis's observations, detailed in her opinion, revealed a process where an agent prompted ChatGPT to compile a narrative for a report using only a brief descriptive sentence and several images. This method, she noted, undermined the agents’ credibility and potentially explained the "inaccuracy of these reports." Crucially, the judge identified factual discrepancies between the official narratives produced by AI and what was visually documented in body camera footage, highlighting a significant disconnect between AI-generated accounts and reality.

Experts in the field have strongly condemned this specific application of AI. Ian Adams, an assistant criminology professor at the University of South Carolina and a member of the Council for Criminal Justice's AI task force, described the agent's actions as "the worst of all worlds." He elaborated that providing a single sentence and a few pictures to generate a critical report "goes against every bit of advice we have out there" and represents a "nightmare scenario." The core issue, experts argue, is that use-of-force reports rely heavily on an officer’s specific perspective and actual experience to meet the judicial standard of "objective reasonableness." Using AI without this direct human input risks the program making up facts in high-stakes situations, rather than reflecting reality.

The Department of Homeland Security has yet to respond to inquiries regarding this practice, and it remains unclear whether the agency has established any guidelines or policies for AI use by its agents. This lack of clear policy is a broader issue, as few law enforcement departments nationwide have implemented comprehensive guidelines. Those that have often prohibit the use of predictive AI, especially for reports that justify critical law enforcement decisions like use-of-force incidents, precisely because these reports demand the specific, articulated events and thoughts of the involved officer.

Beyond accuracy, the use of public AI tools like ChatGPT for such sensitive information raises significant privacy concerns. Katie Kinsey, chief of staff and tech policy counsel at the Policing Project at NYU School of Law, warned that agents using public versions of ChatGPT might unknowingly lose control of uploaded images, allowing them to become part of the public domain and potentially accessible to malicious actors. Kinsey further highlighted a common pattern in law enforcement where new technologies are adopted without prior understanding of risks or established guardrails, leading to reactive policy-making after mistakes have already occurred. She advocated for a proactive approach, emphasizing the importance of transparency and understanding risks before deploying AI, suggesting simple measures like labeling AI-written reports, as seen in Utah and California.

Concerns also extend to the effectiveness of AI in interpreting visual evidence. While some tech companies, such as Axon, offer AI components with body cameras to assist in report writing, these systems typically limit themselves to audio analysis. They avoid using visual inputs because, as companies and experts explain, different AI applications can interpret visual components like colors or facial expressions in vastly different ways, making them unreliable for factual reporting. Andrew Guthrie Ferguson, a law professor at George Washington University Law School, questioned the professionalism of using predictive analytics in such contexts. He articulated the danger that AI might reflect "what the model thinks should have happened, but might not be what actually happened," which is wholly inappropriate for justifying actions in court.

The judge's footnote and subsequent expert commentary underscore a critical juncture for law enforcement. While AI offers potential benefits, its application in sensitive areas like use-of-force reports without proper oversight, clear policies, and a deep understanding of its limitations, poses substantial risks to accuracy, individual privacy, and the fundamental principles of justice and accountability.