AI Hallucination Case Opens New Era for Procurement Litigation

Government Contracts attorney Amy Fuentes was interviewed for a Bloomberg Law article about a federal court decision in a bid protest over an agency's use of artificial intelligence (AI) to evaluate proposals. The U.S. Court of Federal Claims ruled against Trax International Corp.'s protest of a $449 million contract the U.S. Army awarded to a rival company, finding Trax did not sufficiently connect alleged AI hallucinations to errors in evaluation and overall did not adequately demonstrate overreliance on AI tools. Amy told Bloomberg Law the opinion is "important but narrow" because the court required the Army to submit AI-generated evaluations for the administrative record, which "leaves room for a future protest where the record shows evaluators simply adopted AI output without meaningful review." Indeed, she continued, the "practical pressure on agencies to use AI is real" because of the volume of proposals to review and the ability of AI to triage records and streamline evaluation, but awards still have to "reflect human judgment rather than acceptance of an automated AI output." Contractors considering protests may cite Federal Acquisition Regulation (FAR) 15.308 and look to instances in which the record shows hallucinated or incorrect conclusions "simply cut and pasted into the evaluation without meaningful human review," she added, agreeing with other practitioners who forecasted a new phase of procurement litigation focusing on how the government uses AI to grant contracts and awards.

READ: AI Hallucination Case Opens New Era for Procurement Litigation (Subscription required)