TALKING

With fast , natural language and seemingly secure responses , these models are becoming the first choice of many ordinary citizens . But are we aware of the risks built into this new feature ?

According to a recent paper written by researchers at Stanford University , the University of Southern California , Carnegie Mellon University and the Allen Institute for AI , LLMs such as GPT and LLaMA-2 are often reluctant to express uncertainty , even when their answers are incorrect : about 47 % of the answers provided with high confidence by the models were wrong .

For example , a categorical statement about a country ’ s capital might be preferred by note-takers , even if the model was uncertain , resulting in a potentially incorrect but confidently presented answer .

Rodrigo Pereira , CEO , A3Data

In addition , the research addresses the issue of biases in models and human annotation . During the Reinforcement Learning with Human Feedback ( RLHF ) process , language models are trained to optimize responses based on human feedback . However , this process can amplify certain biases present in the training data or in the feedback itself .

Among the biases that must be taken into account are gender and race . In the case of providing feedback with these stereotypes or avoiding expressing uncertainty in contexts that involve minorities , models end up perpetuating and amplifying these human perspectives .

These biases are concerning because they shape the way responses are generated and perceived by users . When combined with the excessive trust that users tend to place in the answers of LLMs , these biases can lead to the spread of distorted information and the consolidation of social biases .

We are , therefore , facing a possible vicious circle . As more people turn to LLMs for information , overreliance on these models can amplify the spread of misinformation .

In this sense , the process of aligning models with human feedback ( RLHF ) may be exacerbating

Another worrying bias is the preference of note-takers for answers that sound more assertive , even when there are uncertainties about this information . This leads models to avoid expressions of doubt to the user , creating the false illusion of sound knowledge , when in fact they may be wrong .

Among the biases that must be taken into account are gender and race .

www . intelligentcio . com INTELLIGENTCIO LATAM 33