6-11X speedup running Qwen3.6 on my RTX 4090, full repo.

2 comments

• shutty 8 minutes ago
  Nice but it would be also great to include strata and llamacpp to the benchmark. Otherwise it’s 11x speedup over the baseline no one uses
• kennywinker 7 minutes ago
  That's cool, but what are you doing that you want to run qwen3.6-35b-a3b, over qwen3.8-flash.next, which runs great on a 4090
  [-]
  • AI4Science 4 minutes ago
    This was a personal project just wanted to see how I can do speed ups and track the process of it! It was more of a project to learn how to systematically do speedup rather than the final results!
  • vikramkr 4 minutes ago
    That would have to be more heavily quantized right? It's a 128bn vs 35bn? How much does that affect it?