0682dae027
This change bring in various interface cleanups along with greatly improving the performance of the sampler. Tested with llama3.2 on local machine. Improves performance from ~ 70 tokens/s -> 135 tokens/s with topK(40) enabled. Without topK performance is ~ 110 tokens/s |
||
|---|---|---|
| .. | ||
| samplers.go | ||
| samplers_benchmark_test.go | ||
| samplers_test.go | ||
| transforms.go | ||
| transforms_test.go | ||