f53f4198c3
There are two benefits to doing this: - Provide a library function that models can use, reducing code for each model implementation - Enables a single place to drop in optimized implementations of attention based on the backend or other factors. One is provided for GGML. On CUDA this improves token generation rate by about 3%. It does not have a significant effect on Metal. Co-authored-by: Daniel Hiltgen <daniel@ollama.com> |
||
|---|---|---|
| .. | ||
| imageproc | ||
| models | ||
| testdata | ||
| model.go | ||
| model_test.go | ||
| process_text.go | ||
| process_text_test.go | ||