The web demo of Alpaca, a small AI language model based on Meta’s LLaMA system, has been taken down offline by researchers at Stanford University due to safety and cost concerns.
Access to large language models containing hundreds or tens of billions of parameters are often restricted to companies that have the resources required to train and run them. Meta planned to share the code for its LLaMA system with select researchers in an attempt to spur research into why language models generate toxic and false text. Meta hoped it could do so without requiring researchers to acquire massive hardware systems.
A group of computer scientists at Stanford University fine-tuned LLaMA to develop Alpaca, an open-source seven-billion-parameter model that reportedly cost less than $600 to build. The code was released last week, and captured the attention of developers after some reportedly managed to get it up and running on Raspberry Pi computers and even a Pixel 6 smartphone.
“Instruction-following models such as GPT-3.5 (text-davinci-003), ChatGPT, Claude, and Bing Chat have become increasingly powerful,” Stanford’s researchers stated.
“Many users now interact with these models regularly and even use them for work. However, despite their widespread deployment, instruction-following models still have many deficiencies: they can generate false information, propagate social stereotypes, and produce toxic language.
“To make maximum progress on addressing these pressing problems, it is important for the academic community to engage. Unfortunately, doing research on instruction-following models in academia has been difficult, as there is no open-source model that comes close in capabilities to closed-source models such as OpenAI’s text-davinci-003.”
Alpaca was fine-tuned with 50,000 text samples guiding the model into following specific instructions to make it function more like to OpenaI’s text-davinci-003.
However the webpage running a demo of Alpaca, which allowed anyone to interact with the model, was taken down shortly after it was launched due to safety issues and rising costs of hosting the model online.
“The original goal of releasing a demo was to disseminate our research in an accessible way We feel that we have mostly achieved this goal, and given the hosting costs and the inadequacies of our content filters, we decided to bring down the demo,” a spokesperson representing Stanford University’s Human-Centered Artificial Intelligence institute, confirmed to The Register in a statement.
Like all other language models, Alpaca is prone to generating misinformation, a property often described as hallucination. Offensive text is another common output.
“Hallucination in particular seems to be a common failure mode for Alpaca, even compared to text-davinci-003,” the researchers noted. In some examples, the model failed to recall the capital of Tanzania correctly and produced false technical information.
Although the web demo has been taken down, the dataset and code describing how to fine-tune the model remain available on GitHub. The researchers said they plan to release details on the model’s weights too.
“Alpaca likely contains many other limitations associated with both the underlying language model and the instruction tuning data. However, we believe that the artifact will still be useful to the community, as it provides a relatively lightweight model that serves as a basis to study important deficiencies,” they said.
“We encourage users to help us identify new kinds of failures by flagging them in the web demo. Overall, we hope that the release of Alpaca can facilitate further research into instruction-following models and their alignment with human values.” ®
