BtMData — Energía y compliance para centros de datos
    All models

    OpenAI · 2023

    GPT-4: training infrastructure breakdown

    The model that defined the GenAI era. A blueprint for what a national AI Factory must deliver.

    Training infrastructure profile

    Public estimates. Not endorsed by the model vendor.
    Parameters

    ~1.76T (MoE, 8 experts × 220B)

    Training tokens

    ~13T tokens

    FLOPs

    ~2.15e25 FLOPs

    Hardware

    ~25,000 NVIDIA A100 (90–100 days)

    GPU-hours

    ~55M GPU-hours

    Estimated cost

    $60M – $100M (training run)

    Cluster power

    ~25 MW (training cluster)

    Sources: SemiAnalysis (Patel & Wong, 2023), Epoch AI, leaked architecture details (George Hotz / Soumith Chintala).

    What this means for your AI infrastructure plan

    Training a model in the same league as GPT-4 requires a dedicated AI factory: thousands of accelerators, megawatts of firmed power, low-PUE cooling, sovereign data residency and a multi-year financing structure. Below is how we translate the public profile above into a deployable plan.

    Cluster sizing

    We size GPU/TPU equivalents to your target FLOPs, accounting for MFU (Model FLOPs Utilization), checkpointing overhead and fault tolerance.

    Power & cooling

    From the ~25 MW (training cluster) reference, we derive substation requirements, PUE targets and cooling architecture (DLC, immersion, hybrid).

    CapEx + OpEx

    Full 20-year financial model: hardware, real estate, energy contracts, staffing, refresh cycles. Stress-tested against energy price scenarios.

    Sovereign deployment

    EU residency, NIS2 / GDPR alignment, optional air-gapped operation. No US CLOUD Act exposure.

    Why this matters

    Planning a model at GPT-4-class scale?

    Talk to our infrastructure team. We size GPU clusters, power, cooling and financing for frontier-scale training and inference workloads — on sovereign EU infrastructure.

    • GPU cluster sizing (H100 / H200 / B200) for your target FLOPs
    • Power, cooling and PUE modelling for 5–150 MW deployments
    • CapEx + 20-year OpEx scenario, financing pathway
    • Site selection across MAD1 (Madrid) + Iberian greenfield + brownfield assets

    Public estimates for GPT-4 indicate the cluster scale, capex and energy footprint required to train a peer model. We translate that into a deployable plan on EU-sovereign infrastructure.

    Other frontier models

    Skip estimates. Get a real sizing.

    We have done this for hyperscale and sovereign customers in Iberia. 24h response.

    Plan a peer model

    All figures on this page are public estimates compiled from third-party research (Epoch AI, SemiAnalysis, MLPerf, vendor press releases) and academic literature. We do not represent the model vendors, and vendors do not endorse these numbers. Use as order-of-magnitude reference only.