pali112/ollama
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Return Correct Prompt Eval Count Regardless of Cache Prompt (#5371)

Browse Source 
* openai compatibility

* Revert "openai compatibility"

This reverts commit d3f98a811e.

* remove erroneous subtraction of prompt cache
This commit is contained in:
royjhan
2024-07-03 13:46:23 -07:00
committed by GitHub
parent daed0634a9
commit 3b5a4a77f3
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
llm/ext_server/server.cpp vendored
View File

@@ -1732,7 +1732,7 @@ struct llama_server_context
slot.n_past -= 1;
}
slot.n_prompt_tokens_processed = slot.n_prompt_tokens - slot.n_past;
slot.n_prompt_tokens_processed = slot.n_prompt_tokens;
if (slot.ga_n != 1)
{