ollama

Commit Graph

Author	SHA1	Message	Date
Michael Yang	bf7edb0d5d	lint linux	2024-06-04 11:13:30 -07:00
Daniel Hiltgen	34b9db5afc	Request and model concurrency This change adds support for multiple concurrent requests, as well as loading multiple models by spawning multiple runners. The default settings are currently set at 1 concurrent request per model and only 1 loaded model at a time, but these can be adjusted by setting OLLAMA_NUM_PARALLEL and OLLAMA_MAX_LOADED_MODELS.	2024-04-22 19:29:12 -07:00

Author

SHA1

Message

Date

Michael Yang

bf7edb0d5d

lint linux

2024-06-04 11:13:30 -07:00

Daniel Hiltgen

34b9db5afc

Request and model concurrency

This change adds support for multiple concurrent requests, as well as
loading multiple models by spawning multiple runners. The default
settings are currently set at 1 concurrent request per model and only 1
loaded model at a time, but these can be adjusted by setting
OLLAMA_NUM_PARALLEL and OLLAMA_MAX_LOADED_MODELS.

2024-04-22 19:29:12 -07:00

2 Commits