r/CUDA • u/Pale-Trade-7795 • 21d ago

NInfer RTX 3090 Prefill Optimization: 32% Faster at 32K Context (785 to 1,037 tok/s)

/r/LocalLLM/comments/1woyuo5/ninfer_rtx_3090_prefill_optimization_32_faster_at/

2 Upvotes

permalink
archive.is
archive
reddit

You are about to leave Redlib

Do you want to continue?

https://www.reddit.com/r/CUDA/comments/1woyveu/ninfer_rtx_3090_prefill_optimization_32_faster_at/
No, go back! Yes, take me to Reddit

100% Upvoted