From 3e353c980b7c7eb7f5d7fde7baa0aaf91bc74208 Mon Sep 17 00:00:00 2001 From: cvrunmin Date: Fri, 28 Nov 2025 14:14:20 +0800 Subject: [PATCH] docs: add infos for split gguf --- docs/api.md | 3 ++- docs/import.mdx | 4 ++++ docs/modelfile.mdx | 12 ++++++++++-- 3 files changed, 16 insertions(+), 3 deletions(-) diff --git a/docs/api.md b/docs/api.md index 99ceaa11a..9bab5dd46 100644 --- a/docs/api.md +++ b/docs/api.md @@ -1176,7 +1176,7 @@ Create a model from: - another model; - a safetensors directory; or -- a GGUF file. +- a GGUF file or directory. If you are creating a model from a safetensors directory or from a GGUF file, you must [create a blob](#create-a-blob) for each of the files and then use the file name and SHA256 digest associated with each blob in the `files` field. @@ -1270,6 +1270,7 @@ A stream of JSON objects is returned: #### Create a model from GGUF Create a model from a GGUF file. The `files` parameter should be filled out with the file name and SHA256 digest of the GGUF file you wish to use. Use [/api/blobs/:digest](#push-a-blob) to push the GGUF file to the server before calling this API. +For a model stored in multiple split GGUF files, includes all split GGUF files in the `files` parameter with the file names and SHA256 digests. It is recommended to provide files in split number order even though Ollama itself will sort them in order. ##### Request diff --git a/docs/import.mdx b/docs/import.mdx index b19596894..26a636564 100644 --- a/docs/import.mdx +++ b/docs/import.mdx @@ -88,6 +88,10 @@ To import a GGUF model, create a `Modelfile` containing: ```dockerfile FROM /path/to/file.gguf ``` +Or: +```dockerfile +FROM /path/to/gguf/split/directory +``` For a GGUF adapter, create the `Modelfile` with: diff --git a/docs/modelfile.mdx b/docs/modelfile.mdx index c91d7310c..1bc28c555 100644 --- a/docs/modelfile.mdx +++ b/docs/modelfile.mdx @@ -12,7 +12,7 @@ A Modelfile is the blueprint to create and share customized models using Ollama. - [FROM (Required)](#from-required) - [Build from existing model](#build-from-existing-model) - [Build from a Safetensors model](#build-from-a-safetensors-model) - - [Build from a GGUF file](#build-from-a-gguf-file) + - [Build from a GGUF file](#build-from-a-gguf-model) - [PARAMETER](#parameter) - [Valid Parameters and Values](#valid-parameters-and-values) - [TEMPLATE](#template) @@ -129,7 +129,7 @@ Currently supported model architectures: - Gemma (including Gemma 1 and Gemma 2) - Phi3 -#### Build from a GGUF file +#### Build from a GGUF model ``` FROM ./ollama-model.gguf @@ -137,6 +137,14 @@ FROM ./ollama-model.gguf The GGUF file location should be specified as an absolute path or relative to the `Modelfile` location. +For GGUF model split into multiple files: + +``` +FROM +``` + +The model directory should contain solely the split GGUF weights of one model. + ### PARAMETER The `PARAMETER` instruction defines a parameter that can be set when the model is run.