introduce
A few days ago, Meta unexpectedly open-sourced their Large Language Model (LLM) Llama 2, a decision that could reshape the current landscape of AI development. It offers an alternative to major players in the space, such as OpenAI and Google, which have decided to keep tight controls on their AI models, limiting accessibility and limiting innovation more broadly. Hopefully, Meta's decision will spark a collective reaction from the open source community, resisting the tendency to limit access to advancements in the field. The new license for Llama 2 goes even further, allowing commercial use, offering developers and businesses the opportunity to leverage the model in existing and new products.
The Llama2 family consists of pretrained and fine-tuned Llama2 and Llama2-Chat, scalable to 70B parameters. These models have been shown to outperform open-source models on various benchmarks [1]. They also insisted against some closed-source models, providing a much-needed boost to the development of open-source AI [2].
If you follow HuggingFace [1]'s Open LLM leaderboard, you'll find that Meta's Llama 2 is firmly in third place. After LLama 2 was released, Stability AI released FreeWilly1 and FreeWilly2 [3]. FreeWilly1 is a fine-tuned version of Llama, and FreeWilly2 is a fine-tuned version of Llama 2. Stability AI says they fine-tuned both models on an Orca-style dataset. The Orca dataset is a large, structured collection of augmented datasets aimed at fine-tuning LLMs, where each entry consists of a question and corresponding response from GPT-4 or GPT-3.5. Why don't we use the FreeWilly2 model? Unfortunately, while Llama 2 allows commercial use, FreeWilly2 can only be used for research purposes and is governed by a non-commercial Creative Commons license (CC BY-NC-4.0).
In this article we also introduce Lla using FastAPI, Celery, Redis and Docker and Meta