On the smaller GPUs, the initial model load of llama2 took over 30s (the default timeout for the DoGenerate helper)
832 B
832 B
On the smaller GPUs, the initial model load of llama2 took over 30s (the default timeout for the DoGenerate helper)