Similar to the llama engine, quantizing the KV cache requires flash attention to be enabled through the Ollama server. |
||
|---|---|---|
| .. | ||
| cache.go | ||
| cache_test.go | ||
| runner.go | ||
Similar to the llama engine, quantizing the KV cache requires flash attention to be enabled through the Ollama server. |
||
|---|---|---|
| .. | ||
| cache.go | ||
| cache_test.go | ||
| runner.go | ||