Templates
Start from a common workload. RA-BIT opens with the model and an example usage filled in; change anything before it sizes the GPUs, time and cost.
- inference
Serve Llama 3.1 70B
A chat endpoint for Llama 3.1 70B in FP8: memory, GPUs and cost per million tokens.
inferencechat
Size in RA-BIT - fine-tuning
Fine-tune 8B with LoRA
Adapt Llama 3.1 8B to your data with LoRA in BF16: GPUs, hours and cost for one run.
fine-tuninglora
Size in RA-BIT - batch
Batch embeddings
Embed a document corpus offline on interruptible GPUs: throughput, time and cost.
batchembeddings
Size in RA-BIT - inference
Serve Mixtral 8x7B
A mixture-of-experts endpoint: 47B parameters stored, about 13B active per token.
inferencemoe
Size in RA-BIT - pretraining
Pretrain a 1B model
Train a small model from scratch on 100B tokens: cluster size, days and cost.
pretrainingresearch
Size in RA-BIT - research
Research GPU pool
A standing pool of GPUs for a research team, busy part of the week.
researchpool
Size in RA-BIT