Fireworks AI is an enterprise-grade inference platform specializing in high-performance serving of open-source and custom AI models, optimized for speed, scalability, and production workloads. Founded by Lin Qiao, the platform processes over 13 trillion tokens daily, sustaining approximately 180,000 requests per second across its globally distributed Inference Cloud.¹⁾

Fireworks AI provides serverless and dedicated inference for a wide range of open-source models, along with tools for fine-tuning, function calling, and compound AI system development. The platform is designed for production teams requiring enterprise-grade reliability, SLA guarantees, and auto-scaling.²⁾

The platform delivers industry-leading throughput and low latency:

• Over 13 trillion tokens processed daily
• Approximately 180,000 requests per second sustained throughput
• 1,000+ tokens per second generation on large models
• Top benchmarks on Artificial Analysis evaluations

Fireworks achieves these speeds through proprietary inference engine optimizations including FireOptimizer for automatic model optimization during deployment.³⁾

The platform provides serverless inference for pre-deployed models including:

• Llama 3.1 (70B Instruct and 405B)
• Mixtral and Mistral variants
• DeepSeek models
• Vision and image models (Stable Diffusion, Flux)
• Speech-to-text (Whisper-v3)

Users can also upload custom base models, fine-tuned weights (including LoRA adapters), and deploy them via the same unified API.

FireFunction provides reliable function calling and structured JSON output from open-source models. This capability enables production-ready agent architectures where consistent tool use and structured outputs are critical for workflow automation.⁴⁾

Fireworks supports building complex AI systems including:

• AI agents with tool integration
• Retrieval-Augmented Generation (RAG) pipelines
• Multi-step workflows combining text, vision, embeddings, image generation, and speech-to-text through a unified API

Fireworks uses per-token billing for serverless inference, scaled by model size. Enterprise plans include SLA-backed uptime, compliance features, and no additional cost for fine-tuning or deploying custom models. New users receive startup credits with pay-as-you-go billing thereafter.⁵⁾

In 2025-2026, Fireworks entered public preview on Microsoft Foundry (Azure), embedding its inference engine for state-of-the-art open models with enterprise governance controls and customization capabilities.⁶⁾

• Together AI
• Replicate
• Groq Inference