GPT-OSS-120B Hardware Requirements: Memory and Capacity Planning
OpenAI's model card describes operation within an 80 GB GPU, while its Ollama guide identifies at least 60 GB of VRAM or unified memory as a starting configuration.
GPT-OSS-120B needs a supported runtime and enough usable memory for its weights plus the workload. OpenAI's model card describes operation within an 80 GB GPU, while its Ollama guide identifies at least 60 GB of VRAM or unified memory as a starting configuration. These are documented deployment references, not guarantees that every 60 GB or 80 GB machine will serve your context length and user load.
Do not translate either statement into “80 GB of RAM is sufficient.” Dedicated GPU memory, host RAM and unified memory are different arrangements. The model representation, supported kernels, offload behavior and simultaneous requests determine whether a configuration is practical.
Start with a Documented Configuration
| Reference | What it supports | What remains to test |
|---|---|---|
| Publisher's model card | An 80 GB GPU deployment example for the released model. | Your runtime version, context, output budget and concurrency. |
| OpenAI's Ollama guide | At least 60 GB VRAM or unified memory guidance; CPU offload changes performance. | Available memory after other applications and latency under the intended workload. |
| OpenAI's Transformers guide | An MXFP4-aware execution path with specific accelerator and kernel assumptions. | Whether the selected hardware and dependency versions use that path or a different representation. |
These references describe different software paths. A GPU generation supported by one optimized kernel does not establish support in every runtime. Likewise, a machine with unified memory should be evaluated through a supported backend for that platform, not by copying an NVIDIA-specific installation recipe.
GPT-OSS-120B is an open-weight mixture-of-experts model under Apache 2.0. Only a subset of experts participates in processing a token, but the remaining weights still need to be available to the implementation. Active-parameter counts are not a replacement for the total memory budget. The model uses the Harmony message format; a compatible template and parser are part of correct execution.
Build the Memory Budget in Separate Parts
Budget for resident weights + context-related state + compute workspace + runtime overhead + operating headroom. Do not assign all installed memory to the first term. Other processes and the operating environment consume resources, and runtime allocations can change with workload.
The released checkpoint uses MXFP4 for its mixture-of-experts weights, rather than storing every tensor in one uniform four-bit format. Quantization metadata and other tensors matter. A simple parameter-count calculation can illustrate why full-precision weights are larger, but it cannot replace the artifact's documented layout or a runtime measurement.
Longer inputs and generated outputs increase request-state requirements. Concurrent requests add another dimension. A successful short prompt at concurrency one therefore proves much less than stable service at peak demand. Reserve output capacity as well as input capacity; setting a large context maximum without testing it can make an otherwise usable deployment fail.
Record storage separately. Downloads, converted artifacts, caches and retained revisions consume disk space. SSD capacity affects what you can store and load, but it does not become fast GPU memory merely because a runtime can move data between devices. CPU offload may make an experiment possible while producing unacceptable interactive latency.
Choose the Runtime Before Buying Hardware
Ollama and LM Studio can simplify local model management through their supported backends. Transformers offers a lower-level Python integration path. Server-oriented runtimes may add batching and distributed execution options. Select the path that matches the task, then confirm architecture, weight representation, accelerator, driver and message-format support together.
Do not combine installation steps from these alternatives into one mandatory stack. Using a desktop runtime does not automatically require a separate Transformers environment or the openai-harmony package. Conversely, installing a tokenizer or message-format library does not supply compatible GPU kernels, authentication or a working application server.
For multi-GPU hardware, inspect how the chosen runtime distributes the model and request state. Aggregate VRAM is not sufficient evidence: per-device allocations, communication, interconnect bandwidth and supported parallelism matter. Measure the complete request path. A nominally larger configuration can have worse interactive behavior if data movement becomes the bottleneck.
Keep inference and fine-tuning separate. Training introduces gradients, optimizer state and activations, depending on the method. A documented inference configuration is not a promise that full-model fine-tuning fits on the same hardware. Evaluate any adaptation method with its own implementation-specific resource plan.
Test a Workload Envelope Before Committing
Consider a hypothetical internal document assistant. One workload is a short question with a small passage; another supplies several document sections. Define expected answer content, input and output limits, response-time targets and request arrival patterns before comparing hardware. The longer case must not disappear from the test simply because the short case works.
| Record | Required detail | Decision it supports |
|---|---|---|
| Configuration | Model revision, representation, runtime, backend and device allocation. | Can the result be reproduced? |
| Workload | Input/output limits, task mix and active requests. | Does the test represent expected demand? |
| Memory | Peak device and host usage, offload and failures. | Is there sufficient operating headroom? |
| Timing | Queue time, first-token time and completion time. | Does the user receive a usable answer soon enough? |
| Quality | Accepted answers, unsupported claims and reviewer corrections. | Is the extra capacity producing useful work? |
| Failure handling | Cancellation, timeout, overload and recovery behavior. | Can the service fail without losing control? |
Begin with one request and verify the returned final answer. Then increase input size and concurrency in controlled steps. Record failed and canceled requests, not just successful runs. Stop increasing load when a defined quality, latency or memory limit is exceeded. The outcome is a measured operating envelope for that configuration, not a universal benchmark.
Compare GPT-OSS-20B on the same permitted tasks when it fits the business requirement. Select 120B when its observed benefit justifies the additional resources and operating burden. A larger model can still fail a poorly specified task or one lacking relevant evidence; purchasing memory does not repair those problems.
Account for Serving, Privacy and Recovery
A local model that responds to a prompt is not yet a managed service. Before opening access to a team, define authentication, permitted clients, request-size limits, concurrency limits, logging and data retention. Keep the initial endpoint restricted. Do not expose an unauthenticated inference service simply to make a demonstration easier.
Separate model output from tool execution. A generated tool request must pass application validation and the appropriate approval boundary. Harmony parsing describes message structure; it does not authorize a filesystem change, database operation or remote request. Test malformed output and client disconnects as well as the normal response.
Plan startup, restart and model availability. If loading fails, surface a clear error rather than silently routing confidential input to a cloud provider. If a request times out, determine whether computation was actually canceled. Retained logs should support troubleshooting without unnecessarily storing complete sensitive prompts.
Make the Purchase Decision from Accepted Work
Compare configurations using total cost per accepted task, including hardware allocation, power, maintenance and review. Keep latency and critical quality requirements as pass/fail conditions rather than letting a low cost compensate for unusable answers. Retest after changing a runtime, model representation or workload.
The decision record should state the chosen hardware and software, workload limits, observed capacity, unresolved issues and responsible operator. No hardware trial or throughput result is claimed in this guide. For the surrounding application, access and integration design, see Cognativ's AI implementation services.
