California Supreme Court Demands Answers on AI Use in Bar Exam Questions

California’s Supreme Court demands answers from the State Bar after learning AI was used without approval to draft questions for the February 2025 bar exam.

Key points:

  • California Supreme Court was unaware AI had been used to craft February 2025 bar exam questions.
  • The court demands full disclosure from the State Bar on the use, reliability, and oversight of AI-generated questions.
  • State Bar leadership acknowledged a communication breakdown and structural reforms are underway.

California’s Supreme Court is demanding answers from the State Bar after discovering that artificial intelligence was used to draft some of the questions on the February 2025 bar exam—a fact the justices only learned through a State Bar press release issued earlier this week.

A court spokesperson confirmed that the seven justices had no prior knowledge of the State Bar’s use of AI to help create 23 out of the 200 multiple-choice questions on the exam. In response, the court has requested a full explanation from the State Bar regarding how and why AI was used, what steps were taken to ensure the questions’ reliability, and whether any AI-generated questions were excluded from scoring for being unreliable, according to The Recorder.

The request comes amid broader concerns about the integrity of the February exam, which was already marred by widespread technical problems. The justices’ concerns will be addressed in a pending petition related to lowering the raw passing score for the exam cohort. The court has asked for the State Bar’s full explanation to accompany that petition.

In a statement issued Tuesday, the State Bar said that the decision to allow AI drafting was made internally by staff within the Admissions Department without adequately informing State Bar leadership. “This was a breakdown,” the Bar acknowledged, adding that structural changes have been made to address the lapse. However, the Bar declined to disclose which AI platform was used or how it was trained to generate questions suitable for assessing minimal competency to practice law in California.

The Bar insisted that there was no conflict of interest between ACS Ventures—its psychometric vendor—both drafting and validating exam questions, stating that the validation process is statistical and objective regardless of a question’s source. Still, the revelation has triggered significant concern.

Alex Chan, chair of the Bar’s Committee of Bar Examiners, emphasized that neither he nor the committee had been consulted or approved the use of AI in drafting the February exam content. “While the court last year directed the Committee of Bar Examiners to explore broader reforms to the bar exam, it did so in a narrow context,” Chan said. “The court has not endorsed the use of AI beyond this narrow context or authorized such use in drafting bar exam content.”

Chan indicated that the committee would launch its own investigation into how AI usage occurred without approval. “Any future use of AI—if deemed appropriate—must be accompanied by strict oversight, independent validation, and approval by the California Supreme Court,” he said.

The results for the February 2025 bar exam are still scheduled to be released on May 2. 

Legal.io Logo
Welcome to Legal.io

Connect with peers, level up skills, and find jobs at the world's best in-house legal departments