docs: add infos for split gguf
This commit is contained in:
parent
10dc89faca
commit
3e353c980b
|
|
@ -1176,7 +1176,7 @@ Create a model from:
|
|||
|
||||
- another model;
|
||||
- a safetensors directory; or
|
||||
- a GGUF file.
|
||||
- a GGUF file or directory.
|
||||
|
||||
If you are creating a model from a safetensors directory or from a GGUF file, you must [create a blob](#create-a-blob) for each of the files and then use the file name and SHA256 digest associated with each blob in the `files` field.
|
||||
|
||||
|
|
@ -1270,6 +1270,7 @@ A stream of JSON objects is returned:
|
|||
#### Create a model from GGUF
|
||||
|
||||
Create a model from a GGUF file. The `files` parameter should be filled out with the file name and SHA256 digest of the GGUF file you wish to use. Use [/api/blobs/:digest](#push-a-blob) to push the GGUF file to the server before calling this API.
|
||||
For a model stored in multiple split GGUF files, includes all split GGUF files in the `files` parameter with the file names and SHA256 digests. It is recommended to provide files in split number order even though Ollama itself will sort them in order.
|
||||
|
||||
##### Request
|
||||
|
||||
|
|
|
|||
|
|
@ -88,6 +88,10 @@ To import a GGUF model, create a `Modelfile` containing:
|
|||
```dockerfile
|
||||
FROM /path/to/file.gguf
|
||||
```
|
||||
Or:
|
||||
```dockerfile
|
||||
FROM /path/to/gguf/split/directory
|
||||
```
|
||||
|
||||
For a GGUF adapter, create the `Modelfile` with:
|
||||
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ A Modelfile is the blueprint to create and share customized models using Ollama.
|
|||
- [FROM (Required)](#from-required)
|
||||
- [Build from existing model](#build-from-existing-model)
|
||||
- [Build from a Safetensors model](#build-from-a-safetensors-model)
|
||||
- [Build from a GGUF file](#build-from-a-gguf-file)
|
||||
- [Build from a GGUF file](#build-from-a-gguf-model)
|
||||
- [PARAMETER](#parameter)
|
||||
- [Valid Parameters and Values](#valid-parameters-and-values)
|
||||
- [TEMPLATE](#template)
|
||||
|
|
@ -129,7 +129,7 @@ Currently supported model architectures:
|
|||
- Gemma (including Gemma 1 and Gemma 2)
|
||||
- Phi3
|
||||
|
||||
#### Build from a GGUF file
|
||||
#### Build from a GGUF model
|
||||
|
||||
```
|
||||
FROM ./ollama-model.gguf
|
||||
|
|
@ -137,6 +137,14 @@ FROM ./ollama-model.gguf
|
|||
|
||||
The GGUF file location should be specified as an absolute path or relative to the `Modelfile` location.
|
||||
|
||||
For GGUF model split into multiple files:
|
||||
|
||||
```
|
||||
FROM <model directory>
|
||||
```
|
||||
|
||||
The model directory should contain solely the split GGUF weights of one model.
|
||||
|
||||
### PARAMETER
|
||||
|
||||
The `PARAMETER` instruction defines a parameter that can be set when the model is run.
|
||||
|
|
|
|||
Loading…
Reference in New Issue