Jesse Gross d5a0d8d904 llm: New memory management ... This changes the memory allocation strategy from upfront estimation to tracking actual allocations done by the engine and reacting to that. The goal is avoid issues caused by both under-estimation (crashing) and over-estimation (low performance due to under-utilized GPUs). It is currently opt-in and can be enabled for models running on the Ollama engine by setting OLLAMA_NEW_ESTIMATES=1. Behavior in other cases is unchanged and will continue to use the existing estimates.		2025-08-14 15:24:01 -07:00
..
.gitignore	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0001-ggml-backend-malloc-and-free-using-the-same-compiler.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0002-pretokenizer.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0003-clip-unicode.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0004-solar-pro.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0005-fix-deepseek-deseret-regex.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0006-maintain-ordering-for-rules-for-grammar.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0007-sort-devices-by-score.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0008-add-phony-target-ggml-cpu-for-all-cpu-variants.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0009-remove-amx.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0010-fix-string-arr-kv-loading.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0011-ollama-debug-tensor.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0012-add-ollama-vocab-for-grammar-support.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0013-add-argsort-and-cuda-copy-for-i32.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0014-graph-memory-reporting-on-failure.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0015-ggml-Export-GPU-UUIDs.patch	update vendored llama.cpp and ggml (#11823)	2025-08-14 14:42:58 -07:00
0016-add-C-API-for-mtmd_input_text.patch	llm: New memory management	2025-08-14 15:24:01 -07:00
0017-no-power-throttling-win32-with-gnuc.patch	llm: New memory management	2025-08-14 15:24:01 -07:00
0018-BF16-macos-version-guard.patch	llm: New memory management	2025-08-14 15:24:01 -07:00
0019-Enable-CUDA-Graphs-for-gemma3n.patch	llm: New memory management	2025-08-14 15:24:01 -07:00
0020-Disable-ggml-blas-on-macos-v13-and-older.patch	llm: New memory management	2025-08-14 15:24:01 -07:00
0021-fix-mtmd-audio.cpp-build-on-windows.patch	llm: New memory management	2025-08-14 15:24:01 -07:00
0022-ggml-No-alloc-mode.patch	llm: New memory management	2025-08-14 15:24:01 -07:00