There are two cases where we may not have an output after computing: - Prompt processing where the length of the input exceeds the batch size - Internal memory management operations such as cache defrag and shift |
||
|---|---|---|
| .. | ||
| backend | ||
| nn | ||
| backend.go | ||
There are two cases where we may not have an output after computing: - Prompt processing where the length of the input exceeds the batch size - Internal memory management operations such as cache defrag and shift |
||
|---|---|---|
| .. | ||
| backend | ||
| nn | ||
| backend.go | ||