ollama

Commit Graph

Author	SHA1	Message	Date
Daniel Hiltgen	31f64183dc	perf: build graph for next batch in parallel to keep GPU busy This refactors the main run loop of the ollama runner to perform the main GPU intensive tasks (Compute+Floats) in a go routine so we can prepare the next batch in parallel to reduce the amount of time the GPU stalls waiting for the next batch of work.	2025-08-19 12:33:03 -07:00
Daniel Hiltgen	949b6c01e0	Revamp go based integration tests This uplevels the integration tests to run the server which can allow testing an existing server, or a remote server.	2024-03-23 14:24:18 +01:00

Author

SHA1

Message

Date

Daniel Hiltgen

31f64183dc

perf: build graph for next batch in parallel to keep GPU busy

This refactors the main run loop of the ollama runner to perform the main GPU
intensive tasks (Compute+Floats) in a go routine so we can prepare the next
batch in parallel to reduce the amount of time the GPU stalls waiting for the
next batch of work.

2025-08-19 12:33:03 -07:00

Daniel Hiltgen

949b6c01e0

Revamp go based integration tests

This uplevels the integration tests to run the server which can allow
testing an existing server, or a remote server.

2024-03-23 14:24:18 +01:00

2 Commits