If flash attention is enabled without KV cache quanitization, we will currently always get this warning: level=WARN source=server.go:226 msg="kv cache type not supported by model" type="" |
||
|---|---|---|
| .. | ||
| ggml.go | ||
| ggml_test.go | ||
| gguf.go | ||
| gguf_test.go | ||
| type.go | ||
If flash attention is enabled without KV cache quanitization, we will currently always get this warning: level=WARN source=server.go:226 msg="kv cache type not supported by model" type="" |
||
|---|---|---|
| .. | ||
| ggml.go | ||
| ggml_test.go | ||
| gguf.go | ||
| gguf_test.go | ||
| type.go | ||