After three years of a study focused on the current bar examination and its practices in the legal ecosystem, NCBE set out to work on a pilot test in 2021. The pilot test was designed to get a better assessment of the skills and knowledge of those taking the test; including newly licensed lawyers and those looking to get relicensed. 

Overview of the Pilot Test

The National Conference of Bar Examiners pilot test was conducted in four separate administrations, all between August 2022 and April 2023, and it involved over 2,500 law students and recently licensed lawyers.

The participants were asked to answer drafts of the new examination questions that were being developed for the NextGen examination. After answering these questions, the law students and newly licensed lawyers were asked to provide feedback on both the questions and their overall experience.

The feedback phase of the test is one of three stages of implementation research. The other stages included field testing and a thorough full-length prototype exam. With all three stages completed, subsequent research phases have already started, the first of which was a field test administered in January 2024.

With the administration of the field test completed a full-length prototype test which mimics the real NextGen exam will be administered in October 2024. All the summaries of the research briefs will be published after all phases are completed.

Key Findings of the NCBE Pilot Test

One of the key questions NCBE researchers were seeking answers to was whether examinees would benefit from access to legal resources such as the Federal Rules of Evidence (FRE) during the exam. The results from the pilot test showed that participants who had access took longer to answer all the relevant questions without a corresponding increase in scores.

For this reason, NCBE decided that extensive legal reference materials, such as the FRE, will not be provided on the NextGen exam. However, more data will still be needed to give more question types which focus on performance tasks and questions on foundational concepts and principles.

Time management is also an aspect of concern. NCBE is looking to determine the appropriate amount of time examinees need to answer the questions. According to the pilot results, participants took an average of 1.1 minutes on selected-response questions, approximately 4 minutes on short-answer questions, and around 20 minutes on a set of questions related to a common scenario.

With FRE and time out of the way, researchers also looked into performance differences among examinee groups. The NextGen questions are focused on reducing performance differences between different groups of examinees.

The pilot test showed that the new question types could potentially reduce performance gaps between groups such as men and women. NCBE psychometric experts will continue to study these results in subsequent research phases to ensure fairness and reliability in the exam.

When it comes to the feasibility and cognitive process of administering and scoring the new question types, the NCBE found the new formats practical, engaging, and more skills-oriented than the current bar exam questions. The cognitive processes tested found that participants relied on legal reasoning and knowledge, particularly skills acquired through clinical and internship experiences.

Next Steps

Once the testing phase is completed by the end of 2024, all gathered data will be re-evaluated to determine the right way forward. With that done, the full NextGen bar exam is set to be administered for the first time in July 2026.

So far, 19 jurisdictions have announced plans to adopt the NextGen bar exam, which will replace the current Uniform Bar Examination (UBE). The new exam is centered around nine areas of legal doctrine and seven foundational lawyering skills, along with tenets of attorney ethics.

This will reflect on the shifting areas in both legal education and practice. It will also push an emphasize on skill and knowledge needed in both litigation and transactional legal practice. The exam will fully represent the evolution of how the legal profession assesses the readiness of new attorneys.