Author: admin

A new study by researchers at Google DeepMind and University College London reveals how large language models (LLMs) form, maintain and lose confidence in their answers. The findings reveal striking similarities between the cognitive biases of LLMs and humans, while also highlighting stark differences.

The research reveals that LLMs can be overconfident in their own answers yet quickly lose that confidence and change their minds when presented with a counterargument, even if the counterargument is incorrect. Understanding the nuances of this behavior can have direct consequences on how you build LLM applications, especially conversational interfaces that span several turns.

SOURCE: Venture Beat

Testing confidence in LLMs

A critical factor in the safe deployment of LLMs is that their answers are accompanied by a reliable sense of confidence (the probability that the model assigns to the answer token). While we know LLMs can produce these confidence scores, the extent to which they can use them to guide adaptive behavior is poorly characterized. There is also empirical evidence that LLMs can be overconfident in their initial answer but also be highly sensitive to criticism and quickly become underconfident in that same choice.

To investigate this, the researchers developed a controlled experiment to test how LLMs update their confidence and decide whether to change their answers when presented with external advice. In the experiment, an “answering LLM” was first given a binary-choice question, such as identifying the correct latitude for a city from two options. After making its initial choice, the LLM was given advice from a fictitious “advice LLM.” This advice came with an explicit accuracy rating (e.g., “This advice LLM is 70% accurate”) and would either agree with, oppose, or stay neutral on the answering LLM’s initial choice. Finally, the answering LLM was asked to make its final choice.

A key part of the experiment was controlling whether the LLM’s own initial answer was visible to it during the second, final decision. In some cases, it was shown, and in others, it was hidden. This unique setup, impossible to replicate with human participants who can’t simply forget their prior choices, allowed the researchers to isolate how memory of a past decision influences current confidence.

A baseline condition, where the initial answer was hidden and the advice was neutral, established how much an LLM’s answer might change simply due to random variance in the model’s processing. The analysis focused on how the LLM’s confidence in its original choice changed between the first and second turn, providing a clear picture of how initial belief, or prior, affects a “change of mind” in the model.

Overconfidence and underconfidence

The researchers first examined how the visibility of the LLM’s own answer affected its tendency to change its answer. They observed that when the model could see its initial answer, it showed a reduced tendency to switch, compared to when the answer was hidden. This finding points to a specific cognitive bias. As the paper notes, “This effect – the tendency to stick with one’s initial choice to a greater extent when that choice was visible (as opposed to hidden) during the contemplation of final choice – is closely related to a phenomenon described in the study of human decision making, a choice-supportive bias.”

The study also confirmed that the models do integrate external advice. When faced with opposing advice, the LLM showed an increased tendency to change its mind, and a reduced tendency when the advice was supportive. “This finding demonstrates that the answering LLM appropriately integrates the direction of advice to modulate its change of mind rate,” the researchers write. However, they also discovered that the model is overly sensitive to contrary information and performs too large of a confidence update as a result.

Interestingly, this behavior is contrary to the confirmation bias often seen in humans, where people favor information that confirms their existing beliefs. The researchers found that LLMs “overweight opposing rather than supportive advice, both when the initial answer of the model was visible and hidden from the model.” One possible explanation is that training techniques like reinforcement learning from human feedback (RLHF) may encourage models to be overly deferential to user input, a phenomenon known as sycophancy (which remains a challenge for AI labs).

Implications for enterprise applications

This study confirms that AI systems are not the purely logical agents they are often perceived to be. They exhibit their own set of biases, some resembling human cognitive errors and others unique to themselves, which can make their behavior unpredictable in human terms. For enterprise applications, this means that in an extended conversation between a human and an AI agent, the most recent information could have a disproportionate impact on the LLM’s reasoning (especially if it is contradictory to the model’s initial answer), potentially causing it to discard an initially correct answer.

Fortunately, as the study also shows, we can manipulate an LLM’s memory to mitigate these unwanted biases in ways that are not possible with humans. Developers building multi-turn conversational agents can implement strategies to manage the AI’s context. For example, a long conversation can be periodically summarized, with key facts and decisions presented neutrally and stripped of which agent made which choice. This summary can then be used to initiate a new, condensed conversation, providing the model with a clean slate to reason from and helping to avoid the biases that can creep in during extended dialogues.

As LLMs become more integrated into enterprise workflows, understanding the nuances of their decision-making processes is no longer optional. Following foundational research like this enables developers to anticipate and correct for these inherent biases, leading to applications that are not just more capable, but also more robust and reliable.

Well, that didn’t take long. Just a few days after BMW M announced that a street-legal Track Package is being developed for the M2 coupe, spy shots of the pre-production prototype testing around the Nürburgring have already been snapped.

As the M2 Track Package isn’t due to be launched until 2026, official details are being kept under heavily camouflaged wraps. It’s not even clear whether these new components will be sold individually or as a complete Track Day Package. We do know, however, that this project is being overseen by BMW’s M Performance Parts accessories division. This “serious upgrade” will then probably include M Division-developed tweaks to the suspension, the aerodynamics, the cabin, and possibly the powertrain.

Related

BMW Is Taking The M2 Back To The Nurburgring

BMW’s Street Legal M2 Racing scheduled for 2026

M-Grade Aero And Suspension Upgrades Expected

There are a few details we can glean from these spy shots. That new and enormous rear spoiler, for example, features a Swan Neck mount. This is a notable update over the conventionally-mounted carbon wing offered as part of the first Performance Package BMW launched for the G87 M2 three years ago. Why is that significant? Well, the Swan Neck is a more effective way to force airflow over the wing, and thus improves mid-corner stability.

Also, in 2022, that carbon wing could be traded out for a tail-lip spoiler instead, an option BMW may well introduce for 2026 as well. The new and presumably carbon-fiber front splitter, meanwhile, appears noticeably closer to the track surface in these spy shots. This suggests that an adjustable M Performance suspension has been mounted to this M2 mule, lowering the ride height and dialing in racier handling in the process.

Related

BMW iX3 Test Mule Spied With Some Interesting Details

Is a BMW iX3 M already in the works?

There are some significant differences between this camouflaged bodywork and BMW’s 2022 performance package, too. The side skirt and front bumper aero flicks are nowhere to be seen on the Sao Paulo Yellow panels, suggesting there are limited aero changes to the doors, the side sills and the roofline. The carbon side spats behind the front and rear wheels are also missing. The camouflaged front and rear fenders, meanwhile, could be hiding a wider and/or more muscular wheel arch design. This suggests that the multi-spoke, and presumably lighter-weight, alloys shown on BMW’s official teaser image will be wider than those on the stock M2 to help improve traction. Whether these CS-esque wheels also conceal larger M Performance brakes remains to be seen.

Track Pack Unlikely To Feature Stripped-Out Cabin

It’s highly unlikely that this street-legal track package will feature a stripped-out cabin. Take a look through the rear window, and you’ll notice the rear headrests are still there. More than likely, BMW M Performance Parts have instead opted for a more dynamic carbon fiber and Alcantara-upholstered cabin. In a bid to shed further weight, BMW may have taken inspiration from the F87 M2 Performance Edition, a zestier package that ripped out the dual-zone climate control and power seats, among other cabin comforts, to shed weight and improve maneuverability.

Related

The Best Modern BMW M Car Is Now Available For $50,000

The best modern M car, offering a powerful inline-six paired with a manual transmission, can be had for roughly $50,000.

Whether BMW has upgraded the M2’s 3.0-liter TwinTurbo straight-six also remains to be seen. Indeed, the Bavarian brand may wish to leave a 50-horsepower window between the stock, 473-horsepower M2 Coupe and the 523-hp M2 CS for future upgraded models. In that case, this new track package could emulate its Performance Parts forebears by raising the electronically limited 155-mph top speed. Hopefully more updates will follow in the coming weeks.