Mozilla's Llamafile allows distributing and running large language models (LLMs) as a single file. Llamafile aims to make open-source LLMs more accessible to developers and users. Llamafile supports a variety of models, CPUs and GPUs, and other options.
To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark llamafile.
OpenBenchmarking.org metrics for this test profile configuration based on 80 public results since 3 April 2024 with the latest data as of 8 May 2024.
Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.
Based on OpenBenchmarking.org data, the selected test / test configuration (Llamafile 0.7 - Test: llava-v1.5-7b-q4 - Acceleration: CPU) has an average run-time of 2 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.
Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 0.3%.
This benchmark has been successfully tested on the below mentioned architectures. The CPU architectures listed is where successful OpenBenchmarking.org result uploads occurred, namely for helping to determine if a given test is compatible with various alternative CPU architectures.
2 Systems - 137 Benchmark Results |
Intel Core i9-14900K - ASUS PRIME Z790-P WIFI - Intel Raptor Lake-S PCH Clear Linux OS 41680 - 6.8.9-1432.native - GNOME Shell 46.1 |
||||||||||||||||||||||||||
1 System - 912 Benchmark Results |
|
1 System - 236 Benchmark Results
|
AMD Ryzen 9 7900 12-Core - ASRockRack 1U4LW-B650/2L2T B650D4U-2L2T/BCM - AMD Device 14d8 Ubuntu 24.04 - 6.8.0-31-generic - GNOME Shell 46.0
|
1 System - 71 Benchmark Results
|
Intel Core i7-13700HX - Notebook PD5x_7xSNC_SND_SNE - Intel Raptor Lake-S PCH ManjaroLinux 23.1.4 - 6.6.25-1-MANJARO - KDE Plasma 5.27.11
|
1 System - 6 Benchmark Results
|
AMD Ryzen 7 7800X3D 8-Core - Gigabyte B650M K - AMD Device 14d8 Fedora Linux 39 - 6.8.5-201.fc39.x86_64 - GNOME Shell 45.5
|
3 Systems - 37 Benchmark Results
|
AMD Ryzen 7 PRO 5850U - LENOVO ThinkPad T14s Gen 2a 20XF004WUS - AMD Renoir Fedora Linux 39 - 6.5.8-300.fc39.x86_64 - GNOME Shell 45.0
|
2 Systems - 47 Benchmark Results
|
Intel Core i9-10980XE - ASRock X299 Steel Legend - Intel Sky Lake-E DMI3 Registers Ubuntu 22.04 - 6.5.0-18-generic - GNOME Shell 42.2
|
3 Systems - 58 Benchmark Results
|
AMD Ryzen Threadripper 3970X 32-Core - ASUS ROG ZENITH II EXTREME - AMD Starship Ubuntu 22.04 - 6.5.0-25-generic - GNOME Shell 42.2
|
4 Systems - 14 Benchmark Results
|
2 x AMD EPYC 9684X 96-Core - AMD Titanite_4G - AMD Device 14a4 Ubuntu 23.10 - 6.9.0-060900rc1daily20240327-generic - GCC 13.2.0
|
3 Systems - 14 Benchmark Results
|
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS Pro WS WRX80E-SAGE SE WIFI - AMD Starship Ubuntu 23.10 - 6.5.0-26-generic - GNOME Shell 45.0
|
3 Systems - 14 Benchmark Results
|
Intel Xeon Silver 4216 - TYAN S7100AG2NR - Intel Sky Lake-E DMI3 Registers Debian 12 - 6.1.0-11-amd64 - X Server
|
4 Systems - 13 Benchmark Results
|
AMD EPYC 7F32 8-Core - ASRockRack EPYCD8 - AMD Starship Debian 12 - 6.1.0-11-amd64 - X Server
|
2 Systems - 14 Benchmark Results
|
AMD EPYC 7551 32-Core - GIGABYTE MZ31-AR0-00 v01010101 - AMD 17h Debian 12 - 6.1.0-10-amd64 - GCC 12.2.0
|
3 Systems - 3 Benchmark Results
|
Intel Core Ultra 7 155H - MTL Swift SFG14-72T Coral_MTH - Intel Device 7e7f Ubuntu 23.10 - 6.9.0-060900rc1daily20240327-generic - GNOME Shell 45.2
|
4 Systems - 3 Benchmark Results
|
AMD EPYC 8534P 64-Core - AMD Cinnabar - AMD Device 14a4 Ubuntu 23.10 - 6.8.1-060801-generic - GNOME Shell 45.2 |
2 Systems - 14 Benchmark Results |
AMD EPYC 7551 32-Core - GIGABYTE MZ31-AR0-00 v01010101 - AMD 17h Debian 12 - 6.1.0-10-amd64 - GCC 12.2.0 |
2 Systems - 47 Benchmark Results |
Intel Core i9-10980XE - ASRock X299 Steel Legend - Intel Sky Lake-E DMI3 Registers Ubuntu 22.04 - 6.5.0-18-generic - GNOME Shell 42.2 |
4 Systems - 14 Benchmark Results |
2 x AMD EPYC 9684X 96-Core - AMD Titanite_4G - AMD Device 14a4 Ubuntu 23.10 - 6.9.0-060900rc1daily20240327-generic - GCC 13.2.0 |
3 Systems - 14 Benchmark Results |
Intel Xeon Silver 4216 - TYAN S7100AG2NR - Intel Sky Lake-E DMI3 Registers Debian 12 - 6.1.0-11-amd64 - X Server |
4 Systems - 13 Benchmark Results |
AMD EPYC 7F32 8-Core - ASRockRack EPYCD8 - AMD Starship Debian 12 - 6.1.0-11-amd64 - X Server |
3 Systems - 58 Benchmark Results |
AMD Ryzen Threadripper 3970X 32-Core - ASUS ROG ZENITH II EXTREME - AMD Starship Ubuntu 22.04 - 6.5.0-25-generic - GNOME Shell 42.2 |
4 Systems - 3 Benchmark Results |
Intel Xeon E E-2488 - Supermicro Super Server X13SCL-F v0123456789 - Intel Raptor Lake-S PCH Ubuntu 24.04 - 6.8.0-11-generic - GNOME Shell 45.3 |
2 Systems - 3 Benchmark Results |
2 x Intel Xeon Max 9480 - Supermicro SYS-221H-TNR X13DEM v1.10 - Intel Device 1bce Ubuntu 23.10 - 6.9.0-060900rc2-generic - GNOME Shell 45.0 |
4 Systems - 3 Benchmark Results |
Intel Core i9-14900K - ASUS PRIME Z790-P WIFI - Intel Device 7a27 Ubuntu 23.10 - 6.8.0-phx - GNOME Shell 45.1 |
4 Systems - 3 Benchmark Results |
AMD Ryzen Threadripper 7980X 64-Cores - System76 Thelio Major - AMD Device 14a4 Ubuntu 24.04 - 6.8.0-11-generic - GNOME Shell 45.3 |
2 Systems - 47 Benchmark Results |
Intel Core i9-10980XE - ASRock X299 Steel Legend - Intel Sky Lake-E DMI3 Registers Ubuntu 22.04 - 6.5.0-18-generic - GNOME Shell 42.2 |
4 Systems - 3 Benchmark Results |
AMD EPYC 8534P 64-Core - AMD Cinnabar - AMD Device 14a4 Ubuntu 23.10 - 6.8.1-060801-generic - GNOME Shell 45.2 |
4 Systems - 3 Benchmark Results |
AMD Ryzen 9 7950X 16-Core - ASUS ROG STRIX X670E-E GAMING WIFI - AMD Device 14d8 Ubuntu 23.10 - 6.7.0-060700-generic - GNOME Shell 45.2 |
3 Systems - 14 Benchmark Results |
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS Pro WS WRX80E-SAGE SE WIFI - AMD Starship Ubuntu 23.10 - 6.5.0-26-generic - GNOME Shell 45.0 |
3 Systems - 3 Benchmark Results |
Intel Core Ultra 7 155H - MTL Swift SFG14-72T Coral_MTH - Intel Device 7e7f Ubuntu 23.10 - 6.9.0-060900rc1daily20240327-generic - GNOME Shell 45.2 |