mirror
/
whisper.cpp
mirror of https://github.com/ggerganov/whisper.cpp
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

vulkan: enable mmvq for q2_k on NVIDIA (llama/17675)

This commit is contained in:
Jeff Bolz 2025-12-05 14:21:57 -06:00 committed by Georgi Gerganov
parent 32ba1ec8e0
commit 7e97d3b069
No known key found for this signature in database
GPG Key ID: 449E073F9DC10735
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
ggml/src/ggml-vulkan/ggml-vulkan.cpp
View File

@ -6948,6 +6948,10 @@ static bool ggml_vk_should_use_mmvq(const vk_device& device, uint32_t m, uint32_
// Quantization overhead is not worth it for small k
switch (device->vendor_id) {
case VK_VENDOR_ID_NVIDIA:
if (src0_type == GGML_TYPE_Q2_K) {
return true;
}
if (k <= 4096) {
return false;
}