f03b8bc51a
When calculating the size of the memory required for a compute graph, we may test multiple graphs - for example a vision encoder and the text model. Since these graphs are never run at the same time, we just want the max size. Typically, a new graph only reallocates memory if it doesn't fit in the existing space, so the last graph reservation is the max size. However, the Vulkan backend imposes a 1G cap for a single allocation, which means that the graph may require multiple allocations. This results in a problem if: - There is an old graph with one small chunk and one big chunk - A new graph with one big chunk that is smaller than the total of the old graph. In this case, the big chunk of the new graph will trigger a reallocation, which will free the old second chunk. The total amount of memory reported will be lower than the max. To avoid this, we should explicitly take the max from each graph. |
||
|---|---|---|
| .. | ||
| backend | ||
| nn | ||
| backend.go | ||
| device.go | ||
| path.go | ||