Meta, the company formerly known as Facebook, has recently unveiled Llama 3, the latest iteration of its large language model. This advanced model is available in two versions: an eight billion (8B) parameter version and a 70 billion (70B) parameter version. In this article, we will explore how to run the 8B parameter version of Llama 3 locally, a more feasible option for standard desktops or laptops that may struggle to run the larger 70B version.
Llama 3’s performance overview
Llama 3 is an impressive large language model. The 8B parameter version, trained using 1.3 million hours of GPU time, outperforms its predecessor, Llama 2, in several ways. For instance, it is 34% better than the 7 billion parameter version of Llama 2 and 14% better than the 13 billion parameter version. Remarkably, the 8B parameter version of Llama 3 even surpasses the performance of the 13 billion parameter version of Llama 2. It only falls short by 8% when compared to the 70B parameter version of Llama 2, making it an impressive model for its size.
The 8B parameter version of Llama 3 has a knowledge cutoff date of March 2023, while the 70B version extends to December 2023. As a result, there will be a real-world information discrepancy between the two.
How to run Llama 3 on your PC
To run Llama 3 on Windows, we will use LM Studio.
- To get started, visit lmstudio.ai and download the appropriate LM Studio version for your system.
- Once downloaded, install LM Studio.
- Open LM Studio.
- Once inside LM Studio, ensure that the Llama 3 model is downloaded. If it’s already installed, it should display on the home page. If it isn’t, use the search bar or search icon on the left-hand vertical bar to find it and then download it.
- Once Llama 3 is installed, click the AI Chat icon on the left-hand vertical bar within LM Studio. This will open a chat interface similar to ChatGPT.
- Click Select a model to load at the top of the page. Select the Llama 3 LLM you just downloaded. LM Studio will load the model, which could take a few seconds.
- You can now chat to Llama 3 on your device.
Notably, if you aren’t using a Windows machine, LM Studio can also be used by Mac owners running new M processors (M1, M2, and M3). There is a beta version available for Linux, too.
How to run Llama 3 using Ollama
Alternatively, you can run Llama 3 locally using the Ollama project. This method will appeal to those who want to run the LLM on a Raspberry Pi.
For Mac and Windows, visit oolama.com and download the version compatible with your operating system. If you’re running Linux (or if you want to install Llama 3 on your Raspberry Pi), simply copy and paste the provided line to download and run the install script. Once installed, you can run Llama 3 directly from the terminal app of your choice.
