Artificial intelligence firm Anthropic has taken an innovative approach to develop a large language model (LLM) that can make value judgments based on user input. Unlike traditional LLMs, which often have preset guardrails to restrict certain outputs, Anthropic’s model, known as “Collective Constitutional AI,” empowers users to influence the values and behavior of the AI system. This unique experiment is a response to concerns that predefined safety measures might limit users’ autonomy and that notions of acceptability and utility can be highly subjective and culturally dependent.
In collaboration with Polis and the Collective Intelligence Project, Anthropic engaged a diverse group of 1,000 users in a groundbreaking initiative. Participants were asked to provide input on a series of questions through polling, with the aim of determining appropriate behavior for the AI model. The challenge was to strike a balance between enabling user-driven value alignment while safeguarding against undesirable outputs.
Anthropic employs a method called “Constitutional AI” to guide the fine-tuning process for LLMs, much like a constitution establishes the fundamental principles of governance in many countries. In this experiment, the goal was to integrate feedback from user groups into the model’s constitution.
According to Anthropic, the experiment was a scientific success, shedding light on the challenges of allowing users to collectively define the values of an LLM product. One of the hurdles the team faced was devising a benchmarking process since this experiment represents a pioneering effort, and no established tests existed for comparing the base model with crowd-sourced values.
Ultimately, the model that incorporated data from user polling feedback marginally outperformed the base model, particularly in mitigating biased outputs. As Anthropic stated in their blog post, “More than the resulting model, we’re excited about the process. We believe that this may be one of the first instances in which members of the public have, as a group, intentionally directed the behavior of a large language model. We hope that communities around the world will build on techniques like this to train culturally- and context-specific models that serve their needs.” This experiment marks a significant step toward more inclusive and user-centric AI development.
Image by freepik
Disclosure Statement: Miami Crypto does not take any external funding, or support to bring crypto news to the readers. We do not have any conflicts of interest while writing news stories on Miami Crypto.