Log In

llm_ai_test [2506132-NE-LLMAITEST18]

Published 23 hours ago9 minute read

AMD Ryzen 7 9700X 8-Core testing with a Gigabyte B650E AORUS ELITE X AX ICE (F36a BIOS) and ASUS NVIDIA GeForce RTX 3080 Ti 12GB on Solus 4.7 via the Phoronix Test Suite.

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128llm_64GB_test_0246810SE +/- 0.00, N = 37.611. (CXX) g++ options: -O3
OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512llm_64GB_test_01326395265SE +/- 0.14, N = 358.141. (CXX) g++ options: -O3
OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024llm_64GB_test_01326395265SE +/- 0.03, N = 357.521. (CXX) g++ options: -O3
OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048llm_64GB_test_01326395265SE +/- 0.19, N = 355.791. (CXX) g++ options: -O3

The test quit with a non-zero exit status. The test quit with a non-zero exit status. The test quit with a non-zero exit status. E: llama-cpp: line 9: ./build/bin/llama-bench: No such file or directory

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128llm_64GB_test_0246810SE +/- 0.00, N = 38.031. (CXX) g++ options: -O3

The test quit with a non-zero exit status. The test quit with a non-zero exit status. The test quit with a non-zero exit status. E: llama-cpp: line 9: ./build/bin/llama-bench: No such file or directory

The test quit with a non-zero exit status. The test quit with a non-zero exit status. The test quit with a non-zero exit status. E: llama-cpp: line 9: ./build/bin/llama-bench: No such file or directory

The test quit with a non-zero exit status. The test quit with a non-zero exit status. The test quit with a non-zero exit status. E: llama-cpp: line 9: ./build/bin/llama-bench: No such file or directory

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512llm_64GB_test_01326395265SE +/- 0.08, N = 357.551. (CXX) g++ options: -O3
OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024llm_64GB_test_01326395265SE +/- 0.22, N = 357.471. (CXX) g++ options: -O3
OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048llm_64GB_test_01326395265SE +/- 0.16, N = 356.051. (CXX) g++ options: -O3

The test quit with a non-zero exit status. The test quit with a non-zero exit status. The test quit with a non-zero exit status. E: llama-cpp: line 9: ./build/bin/llama-bench: No such file or directory

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Text Generation 128llm_64GB_test_01326395265SE +/- 0.03, N = 359.061. (CXX) g++ options: -O3

The test quit with a non-zero exit status. The test quit with a non-zero exit status. The test quit with a non-zero exit status. E: llama-cpp: line 9: ./build/bin/llama-bench: No such file or directory

The test quit with a non-zero exit status. The test quit with a non-zero exit status. The test quit with a non-zero exit status. E: llama-cpp: line 9: ./build/bin/llama-bench: No such file or directory

The test quit with a non-zero exit status. The test quit with a non-zero exit status. The test quit with a non-zero exit status. E: llama-cpp: line 9: ./build/bin/llama-bench: No such file or directory

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512llm_64GB_test_070140210280350SE +/- 0.40, N = 3323.751. (CXX) g++ options: -O3
OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 1024llm_64GB_test_070140210280350SE +/- 0.73, N = 3314.061. (CXX) g++ options: -O3
OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 2048llm_64GB_test_060120180240300SE +/- 0.15, N = 3292.221. (CXX) g++ options: -O3

The test quit with a non-zero exit status. The test quit with a non-zero exit status. The test quit with a non-zero exit status. E: llama-cpp: line 9: ./build/bin/llama-bench: No such file or directory

The test quit with a non-zero exit status. The test quit with a non-zero exit status. The test quit with a non-zero exit status. E: llama-cpp: line 9: ./build/bin/llama-bench: No such file or directory

The test quit with a non-zero exit status. The test quit with a non-zero exit status. The test quit with a non-zero exit status. E: llama-cpp: line 9: ./build/bin/llama-bench: No such file or directory

The test quit with a non-zero exit status. The test quit with a non-zero exit status. The test quit with a non-zero exit status. E: llama-cpp: line 9: ./build/bin/llama-bench: No such file or directory

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 16llm_64GB_test_0510152025SE +/- 0.00, N = 322.88
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 128llm_64GB_test_0510152025SE +/- 0.00, N = 322.84
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 256llm_64GB_test_09001800270036004500SE +/- 0.00, N = 34096
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 512llm_64GB_test_02K4K6K8K10KSE +/- 0.00, N = 38192
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 16llm_64GB_test_0714212835SE +/- 0.00, N = 329.33
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 1024llm_64GB_test_04K8K12K16K20KSE +/- 0.00, N = 316384
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 2048llm_64GB_test_07K14K21K28K35KSE +/- 0.00, N = 332768
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 128llm_64GB_test_0714212835SE +/- 0.00, N = 329.33
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 16llm_64GB_test_03691215SE +/- 0.13, N = 511.81
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 256llm_64GB_test_09001800270036004500SE +/- 0.00, N = 34096
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 512llm_64GB_test_02K4K6K8K10KSE +/- 0.00, N = 38192
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 128llm_64GB_test_03691215SE +/- 0.04, N = 311.89
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 16llm_64GB_test_00.49950.9991.49851.9982.4975SE +/- 0.02, N = 122.22
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 1024llm_64GB_test_04K8K12K16K20KSE +/- 0.00, N = 316384
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 2048llm_64GB_test_07K14K21K28K35KSE +/- 0.00, N = 332768
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 128llm_64GB_test_00.50181.00361.50542.00722.509SE +/- 0.01, N = 32.23
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 256llm_64GB_test_09001800270036004500SE +/- 0.00, N = 34096
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 512llm_64GB_test_02K4K6K8K10KSE +/- 0.00, N = 38192
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 1024llm_64GB_test_04K8K12K16K20KSE +/- 0.00, N = 316384
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 2048llm_64GB_test_07K14K21K28K35KSE +/- 0.00, N = 332768
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 256llm_64GB_test_030060090012001500SE +/- 0.00, N = 31536
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 512llm_64GB_test_07001400210028003500SE +/- 0.00, N = 33072
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 1024llm_64GB_test_013002600390052006500SE +/- 0.00, N = 36144
OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 2048llm_64GB_test_03K6K9K12K15KSE +/- 0.00, N = 312288

llm_64GB_test_0

AMD Ryzen 7 9700X 8-Core @ 5.58GHz (8 Cores / 16 Threads), Motherboard: Gigabyte B650E AORUS ELITE X AX ICE (F36a BIOS), Chipset: AMD Raphael/Granite, Memory: 62GB, Disk: 1000GB Samsung SSD 970 EVO Plus 1TB + 1000GB Samsung SSD 980 PRO 1TB + 2000GB CT2000MX500SSD1 + 480GB SanDisk SSD PLUS + 1024GB Inland SATA SSD, Graphics: ASUS NVIDIA GeForce RTX 3080 Ti 12GB, Audio: NVIDIA GA102 HD Audio, Monitor: 3 x MSI G27C5, Network: Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E

OS: Solus 4.7, Kernel: 6.14.6-319.current (x86_64), Desktop: Budgie 10.9.2, Display Server: X Server, Display Driver: NVIDIA 570.153.02, OpenGL: 4.6.0, Compiler: GCC 15.1.1 20250508 + Clang 19.1.6 + LLVM 19.1.6, File-System: ext4, Screen Resolution: 5760x1080

Kernel Notes: Transparent Huge Pages: always
Environment Notes: __GL_MaxFramesAllowed=1
Compiler Notes: --build=x86_64-solus-linux --disable-libssp --enable-bootstrap --enable-cet --enable-checking=release --enable-default-pie --enable-default-ssp --enable-gnu-indirect-function --enable-host-bind-now --enable-host-pie --enable-languages=ada,c,c++,cobol,fortran,rust --enable-link-serialization --enable-multilib --enable-plugin --enable-shared --enable-threads=yes --target=x86_64-solus-linux --with-arch_32=i686 --with-build-config=bootstrap-lto-lean --with-gcc-major-version-only --with-linker-hash-style=gnu --with-multilib-list=m32,m64
Processor Notes: Scaling Governor: amd-pstate-epp powersave (Boost: Enabled EPP: balance_performance) - CPU Microcode: 0xb404023
Security Notes: gather_data_sampling: Not affected + ghostwrite: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of IBPB on VMEXIT only + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS; IBPB: conditional; STIBP: always-on; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 13 June 2025 20:09 by user money.

Origin:
publisher logo
openbenchmarking
Loading...
Loading...
Loading...

You may also like...