At a recent event hosted by TensorWave in San Francisco for the developer community, MK1 showed an open-source 1M context Llama3-70B model running on AMD MI300X hardware.

There has been much anticipation around AMD’s flagship MI300X accelerator. With unmatched raw specs, the pressing question remains: Can it outperform NVIDIA’s Hopper architecture in real-world AI workloads?

With the release of our new inference engine MK1 Flywheel, we are excited to report that AMD Instinct Series can achieve comparable performance to a compute-matched NVIDIA GPU. We've designed MK1 Flywheel for maximum performance on AMD and NVIDIA chips: our benchmarks demonstrate up to 3.7x higher throughput compared to vLLM.

MK1 has built an optimized LLM inference runtime, called Flywheel, that now blazes with higher throughput and lower latency on NVIDIA Ampere, Ada, Hopper and on AMD Instinct. Try it on Amazon SageMaker and engage with us for large-scale deployment.

We are pushing the performance of AI models to the limit of what’s physically possible. Introducing MK1 Flywheel, our enterprise LLM inference solution. Contact us to become an early partner in the closed beta trial.