What for: fast serverless inference and fine-tuning of 200+ open models behind a single OpenAI-compatible API, no hardware needed.

What it is and who it's for

Together AI is a cloud built around open models: serverless inference for 200+ of them, fine-tuning, and GPU cluster rentals. It's for teams building on open weights that don't want to own hardware and need more than just calling a model — they need to train it for their domain. What stands out is the combination of inference + training + dedicated GPUs behind one API.

Key features

Serverless inference for hundreds of open models (Llama, Qwen, DeepSeek, Mixtral, FLUX and more).
Fine-tuning open LLMs on your own data, with hosting for the result.
Dedicated endpoints and GPU cluster rentals for production or training.
OpenAI-compatible API for an easy migration.
Not just text: image and embedding models too.
Batch inference for cheap bulk processing.

Get started in 5 minutes

Sign up at together.ai and grab an API key.
Call a model through the OpenAI-compatible endpoint:

curl https://api.together.xyz/v1/chat/completions \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"meta-llama/Llama-3.3-70B-Instruct-Turbo","messages":[{"role":"user","content":"Hi"}]}'

To fine-tune, upload a dataset (JSONL) and start a fine-tuning job via their CLI/API.

When to use it, when not to

✅ Use it if you need to fine-tune an open model for your domain and host the result in the same place.
✅ Use it if you're running production on open weights with dedicated endpoints and predictable performance.
❌ Skip it → Groq is better if all you want is maximum inference speed with no training.
❌ Skip it → Replicate is better if the job is running a mix of models (video/audio/images) on a fire-and-forget basis.

Honest pricing

Pay per token for inference (rates vary by model), with fine-tuning and dedicated GPUs billed separately by time. There's a starter credit. Exact numbers are on the provider's site and keep changing.

Gotchas

Fine-tuning needs a clean labeled dataset — garbage in, garbage in the model.
Dedicated GPUs bill by rental time, not by request — a forgotten cluster burns money.
The model lineup and its "Turbo" variants shift; pin versions in your config.
It's a cloud: sensitive data leaves your perimeter — weigh the risk under strict compliance.

🤖 Prompt accelerator

"I want to fine-tune an open model on Together AI for <DESCRIBE THE DOMAIN/TASK>. Help me pick a base model, prepare the dataset in the required JSONL format (show an example structure), estimate how much data I need, launch the fine-tuning job through their API/CLI, and then call the finished model. Also estimate the per-token inference cost."