Video random access memory is the historical expansion of VRAM. In contemporary GPU discussion, the term commonly describes the memory available to an accelerator for working data, even when no video is being rendered. Textures and frame buffers use it in graphics; model parameters, activations and caches use it in machine learning. The label identifies a role in the system, not one universal physical memory technology. [1][2]

A discrete GPU can have dedicated memory connected to its own controllers. Integrated and unified-memory systems may share a physical memory pool with other processors. Those architectures need to be interpreted on their own terms: a shared total must also cover the operating system and other applications. A specification that lists system memory is not automatically a specification for memory exclusively available to one GPU workload. [1][2]

Capacity is measured in bytes and determines how much state can be held. Bandwidth is measured in bytes per second across a defined interface. Latency is the delay associated with an access or transfer. A larger memory can solve an out-of-memory problem without proportionally increasing throughput. A faster interface can move an existing working set more quickly without making a larger working set fit. [3]

Model weights are only one category. Training can also require gradients, optimizer state, saved activations and temporary buffers. Inference typically avoids the normal training state but still needs runtime allocations and intermediate values. Frameworks may reserve blocks of memory for reuse, so memory reported as reserved can differ from the amount occupied by live tensors. Diagnosing a failure requires understanding the allocator as well as the model. [4][5]

For autoregressive language-model inference, the KV cache retains attention-related state from previous tokens. Its size depends on architecture, representation, sequence length and the number of active sequences. Different cache strategies can change allocation and memory use. The memory required to load a checkpoint therefore does not establish the memory required to serve a production distribution of requests. [6]

A practical budget separates persistent state, variable per-request state, temporary workspace and operating headroom. Record which components are measured and which are estimates. Do not add a fixed universal percentage and call it verified capacity: implementation choices can change the relationship substantially. Headroom is useful, but the appropriate margin should come from the observed workload and failure tolerance.

Decimal GB and binary GiB are different units. One GB is one billion bytes; one GiB is 1,073,741,824 bytes. The same byte count can therefore be displayed as different-looking numbers. A discrepancy between a label and a tool's display may reflect units, reserved resources or another allocation boundary. Compare bytes and definitions before concluding that memory is missing. [9]

Quantization changes how some numerical values are represented. A smaller representation can reduce weight or cache storage, but metadata, scaling values and temporary dequantization buffers may remain. Weight-only quantization does not automatically reduce every activation or cache. Quality, supported kernels and hardware behavior must be evaluated together. A four-bit label is not proof that total application memory becomes exactly one quarter of a sixteen-bit configuration. [7]

Reducing batch size or concurrent sequences can lower variable memory use. Offloading can place selected data in host memory or another storage tier, but movement across slower paths can affect latency and throughput. Unified memory mechanisms can simplify addressing and migration; they do not abolish the physical cost of moving data between devices or memory tiers. [6][2]

In data-parallel training, workers commonly hold model replicas while processing different data. More devices can increase aggregate processing capacity without removing a per-device model-fit requirement. Sharding and model-parallel strategies distribute different state, but introduce communication and placement requirements. The relevant memory total depends on the parallelization strategy, not just the sum of capacities on a purchase order. [8]

Check whether capacity is per device, per hardware partition, per node or shared across the system. Identify the memory technology, published bandwidth basis and any partitioning. Then test with the exact model, precision, sequence distribution and concurrency. Record peak allocated and reserved memory during the whole workflow, including startup, warm-up and checkpoint operations where applicable.

The most common mistake is asking how much memory a model needs as though there were one permanent answer. The better question is how much memory this implementation needs for this workload at this quality and latency objective. A capacity figure is a starting constraint; the runtime experiment establishes whether the proposed configuration is usable.