8bf38552de
We currently assign model layers to GPUs according to free VRAM, which assumes that GPU performance is roughly equal. This does not work well for mixed dGPU and iGPU systems because iGPUs typically use system memory which is large but their performance is slow. This instead assigns layers to dGPUs first and then iGPUs. In the future, this could be generalized to have a more fine grained notion of GPU performance but dGPU vs. iGPU performance is the most extreme. |
||
|---|---|---|
| .. | ||
| backend | ||
| nn | ||
| backend.go | ||
| device.go | ||
| path.go | ||