Recommended configuration
How many of which GPU, on which contract, in how many servers, and with which parallel layout.
Launch ·
RA-BIT is available.
Research Acceleration & Benchmarking Intelligence Twin
RA-BIT learns how your AI workloads behave, then determines how and where they should run. It sits between your Compute Plan and the market: from a description of the work to a configuration you can buy, and back again from what really happened.
The closed loop
Plan, model, benchmark, predict, place, run, measure, learn.
Every step feeds the next. Predictions start analytical and become measured as your benchmarks and telemetry come in.
- 01
Plan
Start from a Compute Plan workload in one click, or describe a workload from scratch. Nothing is typed twice.
- 02
Model
RA-BIT builds a workload twin: model, shape, data, traffic, deadline and constraints, with every default labelled.
- 03
Benchmark
Plan a run, get a ready-to-run script, and record the measurement. One run calibrates every other GPU.
- 04
Predict
Runtime, cost, memory and the bottleneck on every accelerator Kovara tracks, as ranges with a confidence breakdown.
- 05
Place
The recommended configuration is matched against live public offers, with a compatibility vector, risk and a fallback.
- 06
Run
Request quotes through the Compute Desk with the configuration filled in, or take it to Kovara Select.
- 07
Measure
Your cluster sends telemetry from DCGM, Prometheus, the PyTorch profiler, NCCL, Slurm or Kubernetes.
- 08
Learn
RA-BIT compares what it predicted with what happened, names the cause of any gap and tells you what to change.
On one page
The six answers every analysis gives.
Each figure has a “Why?” that opens the reasons and an evidence graph: datasheets, the Kovara GPU registry, live market prices, your assumptions, the formulas and any measurement.
Runtime range
Wall-clock time for a training run, or time per output token for serving, as a low–high range.
Cost range
What the completed work costs at today's tracked median and 25th-percentile prices.
Primary bottleneck
Compute, memory bandwidth, memory capacity, communication, storage or CPU, with the evidence.
Confidence
A percentage built from four parts: workload description, hardware facts, performance evidence and market evidence.
Potential optimization
The single change most likely to make the run faster or cheaper, and what to measure next.
How it works
First principles, then your measurements.
RA-BIT’s models are public approximations you can read on every analysis. Measurements replace them as soon as they exist.
Workload twin and fingerprint
Every workload becomes a twin: architecture (dense transformer, mixture of experts, recurrent or state-space, diffusion), size, layers, hidden size, heads, context, batch, data, traffic and latency targets. A fingerprint scores nine dimensions: compute intensity, memory capacity, memory bandwidth, communication, storage I/O, CPU dependence, latency sensitivity, interruption tolerance and scale. Dimensions without data stay empty instead of being guessed.
Bottleneck analysis
Effective speed is the minimum of every limit: P_eff = min(compute, memory, communication, storage, CPU). RA-BIT shows each limit in tokens per second per GPU and marks the one that binds.
Roofline engine
Arithmetic intensity, in FLOPs per byte of memory traffic, is placed against each GPU's ridge point (peak ÷ bandwidth). Left of the ridge, memory bandwidth limits; right of it, compute.
Memory model
Training: weights, gradients and Adam states (16 bytes per parameter for full training), activations with checkpointing, and a workspace reserve. Serving: weights plus the KV cache, which grows with context, or a constant recurrent state for models such as xLSTM.
Communication model
Gradient all-reduce or FSDP exchange inside a server over NVLink, Infinity Fabric or PCIe, and across servers over InfiniBand or Ethernet. Tensor parallelism inside a server is searched automatically.
Prediction with ranges
Analytical estimates come from datasheet peaks and your efficiency assumption, always shown as ranges. A recorded benchmark replaces the estimate on its GPU and calibrates the others, and the confidence rises with it.
Economics and the Pareto frontier
Every option (GPU, count, contract) is priced on completed work. The Pareto frontier shows which options no other beats on both cost and time; six objectives weight the choice, and a deadline filters it.
Placement and risk
The recommendation is matched to live public offers in your geography and compliance scope. Each offer gets a compatibility vector: GPU, contract, interconnect, capacity and evidence grade. Risk is shown as expected cost under interruption scenarios, with a fallback on another accelerator.
What decides it
A sensitivity view moves each assumption down and up and shows whether the GPU choice changes. A break-even line says exactly how fast the runner-up must be to win, so you know which benchmark is worth running.
Learning from telemetry
Predicted and observed throughput are compared on the same configuration. Explicit rules name the cause of a gap: data loading, communication, memory pressure, memory bandwidth, power limits or idle GPUs, each with the measured evidence and a recommendation.
Training compute ≈ 6 · N · D FLOPs (4 · N · D for LoRA) + 6 · L · ctx · d per token for attention Effective speed = min(peak, intensity × bandwidth) × MFU Bottleneck P_eff = min(P_compute, P_memory, P_communication, P_storage, P_cpu) Ridge point peak FLOPS ÷ memory bandwidth KV cache 2 · layers · KV heads · head dim · 2 bytes per token Expected cost direct cost + P(failure) × cost of a failure Break-even runner-up wins when its speed per GPU ÷ the leader's > its price ÷ the leader's price
Inside an analysis
Ten views of one workload.
- Overview
- Workload Twin
- Bottlenecks
- Hardware
- Benchmarks
- Simulation
- Placement
- Economics
- Telemetry
- History
Hardware matrix
The same workload on every accelerator, each at the size RA-BIT would use: fit, GPUs, runtime range, speed per GPU, bottleneck, median price, cost and energy.
Hardware simulator
Change the GPU, the count, the efficiency, the network or the precision and compare with the current answer. Deadline optimization finds the cheapest configuration that finishes in time on each accelerator.
History
Every saved prediction with the prices it used, so a change can be traced to the twin, the market or new evidence.
Specifications
What RA-BIT works with.
- Workloads
- Pretraining, fine-tuning (full or LoRA), inference, research pools and batch jobs
- Accelerators
- B200, H200 SXM, H100 SXM, A100 80GB SXM, L40S, A40, A10, L4, MI300X. Peak throughput, interconnect and compute capability from vendor datasheets; memory, bandwidth and power from the Kovara GPU registry
- Precisions
- BF16, FP8 and INT4 weights; FP8 runs at the BF16 rate on GPUs without FP8 tensor cores
- Parallel layouts
- Data parallel, FSDP and tensor parallel inside a server, chosen per accelerator and GPU count
- Objectives
- Balanced, Cheapest, Fastest, Max reliability, EU sovereign, Lowest energy
- Market data
- Current public offers tracked by Kovara: on-demand, reserved and spot, filtered by geography and SOC 2 or ISO 27001
- Model presets
- Llama 3.1 8B, Llama 3.1 70B, Llama 3.1 405B, Mistral 7B, Mixtral 8x7B, Qwen2.5 7B, Qwen2.5 72B, TiRex-2 (NXAI), from their published configurations
- Benchmarks
- Run IDs such as RB-AX72D91 with a full manifest (model, commit, precision, batch, context, GPU, driver, CUDA, framework, runtime, provider, region); results kept as measured and normalized to tokens per second per GPU
- Telemetry sources
- NVIDIA DCGM, NVML / nvidia-smi, PyTorch profiler, NCCL, Slurm, Kubernetes, Prometheus, or entered by hand
- Metrics read
- Throughput, whole job, Step time, GPU utilization, SM active, Memory bandwidth used, Memory used per GPU, Power per GPU, Data-loader wait, Collective time, Host CPU utilization, P95 latency
- Integrations
- Generated benchmark kit for a planned run, a Prometheus and dcgm-exporter pusher, and a PyTorch or Lightning training callback; an HTTPS ingestion API with analysis-scoped keys
- Known limits
- Pipeline parallelism and mixture-of-experts all-to-all traffic are not modelled yet; every analysis lists its assumptions
For AI teams
Less guessing, fewer idle GPUs.
Stop overprovisioning
See the memory each GPU actually needs and the layout that fits, instead of buying the largest part by default.
Compare on completed work
A cheaper hour is not a cheaper run. RA-BIT compares GPUs on what the finished job costs.
Benchmark only what matters
The break-even line and the sensitivity view point to the one measurement that settles the decision.
Catch slowdowns early
Telemetry against the prediction shows when a run drifts, and why, while there is still time to fix it.
Procurement-ready
Turn the recommendation into a quote request with the GPU, quantity and hours already filled in.
Decisions that stay current
Every save records the prediction and the prices it used. When the market moves, RA-BIT tells you whether the answer changed.
Data you can trust
No invented numbers.
- No tracked offer, no price. Missing inputs, no recommendation: RA-BIT says what it needs.
- Every default is marked as a default, and every assumption is listed on the analysis.
- Recorded benchmark results are final; a wrong one is withdrawn with a reason, never rewritten.
- Analyses belong to your organization. Ingestion keys are stored as hashes, act for one analysis, and stop working when revoked.
Questions
Frequently asked.
Where do the prices come from?
From the current public offers Kovara tracks: active, published and not archived, one price per provider. Where no offer matches, RA-BIT shows no price instead of estimating one.
How accurate are the predictions?
Before any measurement they are analytical estimates from datasheet figures and your efficiency assumption, shown as ranges and labelled so. One benchmark on your own code replaces the estimate on that GPU and narrows every range.
Do I need to know my model's shape?
No. RA-BIT works from the model size alone and marks every default it uses. Choosing a published model fills layers, hidden size and heads from its configuration.
Who can see an analysis?
Only the active members of your organization. Owners and Members can change it; Viewers can read it. Ingestion keys can send data for one analysis and cannot read anything.
What does RA-BIT not model yet?
Pipeline parallelism and expert-parallel traffic in mixture-of-experts models. The page of every analysis lists its assumptions and known limits.
How do I start?
Open RA-BIT from the platform sidebar, or press Analyze with RA-BIT on any workload of your Compute Plan.
Analyze your first workload.
Open RA-BIT from the platform sidebar, or press “Analyze with RA-BIT” on a Compute Plan workload.