kvcache: Log contents of cache when unable to find a slot

There is a bug when using sliding window attention where we run
out of KV cache slots. This is likely due to not correctly removing
all of the entries as they slide out of range. This adds additional
logging when this occurs to track down the source.

Bug #10127
This commit is contained in:
Jesse Gross 2025-08-04 16:44:23 -07:00 committed by Ryan Schumacher
parent c24014a55d
commit 275510ddf5
No known key found for this signature in database
1 changed files with 1 additions and 0 deletions

View File

@ -214,6 +214,7 @@ func (c *Causal) StartForward(ctx ml.Context, batch input.Batch, reserve bool) e
c.curLoc, err = c.findStartLoc()
}
if err != nil {
slog.Warn("unable to find a kv cache slot", "cache", c)
return err
}