Cloud Computing for ML

Almost nothing in this section runs on a laptop in production — pick one cloud, go deep on it, and treat the others as "the same concepts, different names" once you know one well. AWS is the reference here because it's the most common in job postings, not because it's uniquely correct.

Compute

• EC2: raw virtual machines — the building block everything else is built on top of. GPU instances (the p/g families) are where model training and self-hosted inference actually run.
• ECS: AWS's own container orchestration service — simpler than Kubernetes, a reasonable choice when you don't need Kubernetes's full feature set.
• EKS: managed Kubernetes on AWS — see Kubernetes.
• Lambda: serverless functions, billed per invocation — good for lightweight, spiky, stateless workloads (a small preprocessing step, a webhook handler); a poor fit for GPU inference or anything with a long cold-start-sensitive model to load.

aws ec2 run-instances \
  --image-id ami-0abcdef1234567890 \
  --instance-type g5.xlarge \
  --key-name my-key \
  --count 1

aws ec2 describe-instances --filters "Name=instance-type,Values=g5.xlarge" \
  --query "Reservations[].Instances[].[InstanceId,State.Name]"

Storage & Databases

What is object storage, actually? Storage for whole files as opaque "objects" — not a mounted filesystem, and not a database table. Every object lives in a flat namespace addressed by a bucket (a top-level container) and a key (a string identifying the object within it) — the nested "folders" an S3 console shows are a display convention over key prefixes (models/fraud-detector/v3/model.pkl is one flat key, not three real nested directories), not an actual directory tree the way a real filesystem has one.

How does it work? Every read/write is a full HTTP request against the object as a whole — GET/PUT/DELETE a complete object, not a random-access byte-offset seek/write the way a local disk or a mounted block volume works. There's no in-place partial edit: updating part of a file means uploading a full replacement object.

Why is it useful? Virtually unlimited scale, very high durability (S3's standard tier: 11 nines), and decoupled from any single machine — many services can read the same object concurrently over the network, which is exactly why it's the default place datasets, model artifacts, and logs live in a real ML pipeline; nearly every tool in this section (DVC, MLflow, Airflow, Docker registries) can use it as a backend directly.

Limitation: The HTTP-request-per-object model means meaningfully higher per-request latency than local or block storage, and no real random-access read/write within a file — a poor fit for a workload that needs low-latency, in-place reads and writes (that's exactly what block storage, below, or a real database is for).

• S3: object storage — the default place datasets, model artifacts, and logs live; nearly every tool in this section (DVC, MLflow, Airflow, Docker registries) can use S3 as a backend.
• EBS: block storage attached to a single EC2 instance — used for a VM's own disk, not for data meant to be shared across services.
• RDS: managed relational databases (Postgres/MySQL) — see Data Engineering & Versioning.
• DynamoDB: a managed NoSQL key-value/document store — used where Redis-like low-latency lookups need to be durable and fully managed rather than in-memory.

The AWS CLI and boto3 (its Python SDK) for the two things every ML workflow actually does against S3 -- push a model artifact after training, pull it before serving:

aws s3 cp model.pkl s3://my-ml-bucket/models/fraud-detector/v3/model.pkl
aws s3 sync ./training-data s3://my-ml-bucket/datasets/fraud/2026-01/
aws s3 ls s3://my-ml-bucket/models/fraud-detector/

import boto3

s3 = boto3.client("s3")
s3.download_file("my-ml-bucket", "models/fraud-detector/v3/model.pkl", "model.pkl")
s3.upload_file("model.pkl", "my-ml-bucket", "models/fraud-detector/v4/model.pkl")

ML-Specific: SageMaker

AWS's managed ML platform — training jobs, hyperparameter tuning, a built-in model registry, and managed endpoints for serving, all without provisioning the underlying infrastructure by hand. The tradeoff is the same as any managed platform: faster to get running, but more vendor lock-in and less control than assembling the equivalent from EC2 + Docker + a serving tool from APIs & Model Serving yourself. Many teams use SageMaker for training (where its managed job infrastructure saves real time) while self-hosting serving on EKS/Triton for more control over latency and cost.

Networking & Security

• VPC: an isolated virtual network — where every resource above actually lives, with subnets controlling what's public vs. private.
• ALB (Application Load Balancer): distributes incoming traffic across multiple instances/containers — sits in front of a serving fleet.
• API Gateway: manages, authenticates, and rate-limits API traffic in front of Lambda or other backends — see the inference stack for where an API gateway sits in front of LLM serving specifically.
• IAM: identity and access management — who (or what service) is allowed to do what; the single most common source of both "why can't my pipeline read this S3 bucket" bugs and real security incidents (over-permissioned roles).
• Secrets Manager / KMS: managed secret storage and encryption-key management — where API keys and database credentials belong instead of a config file or a Dockerfile.
• ECR: AWS's container registry — see Containers.

A real, minimal IAM policy — a training job's role gets read-only access to exactly one bucket prefix, nothing else:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:ListBucket"],
      "Resource": [
        "arn:aws:s3:::my-ml-bucket",
        "arn:aws:s3:::my-ml-bucket/datasets/fraud/*"
      ]
    }
  ]
}

aws secretsmanager get-secret-value --secret-id prod/model-api/db-credentials \
  --query SecretString --output text

The Pattern to Internalize

Every cloud maps onto the same shape: compute (VMs, containers, serverless), storage (object, block, database), networking (VPC, load balancer, gateway), and identity (IAM). Learning AWS deeply and then encountering GCP or Azure is mostly a vocabulary-mapping exercise rather than learning new concepts from scratch:

GCP for ML

• Vertex AI: Google's unified managed ML platform — training, hyperparameter tuning, a model registry, and managed endpoints, the direct GCP counterpart to SageMaker, with particularly strong integration with Google's own foundation models (Gemini) for teams already building on them.
• GKE (Google Kubernetes Engine): GCP's managed Kubernetes — notably where Kubernetes itself originated conceptually (Google's internal Borg system predates and inspired it), and a common choice for teams wanting the most mature managed Kubernetes experience specifically.
• BigQuery: a serverless, fully-managed data warehouse — genuinely distinctive relative to AWS/Azure's warehouse offerings for how it separates storage and compute and charges per-query rather than per-provisioned-capacity, making it a common choice for large-scale, bursty analytical workloads feeding into ML feature engineering.
• Cloud Storage: GCS, the direct S3 equivalent — the default backend for datasets and model artifacts in a GCP-based stack, same role as S3 in the AWS examples throughout this section.

Azure for ML

• Azure ML: Microsoft's managed ML platform — training pipelines, a model registry, and managed endpoints, Azure's counterpart to SageMaker/Vertex AI, with deep integration into the broader Azure/Microsoft enterprise ecosystem (Active Directory, existing enterprise data estates) that's often the actual reason a team is on Azure in the first place.
• Azure OpenAI: Microsoft's hosted access to OpenAI's models (GPT-4-class and others) through Azure's own infrastructure, identity, and compliance boundary — the common choice for enterprises that need OpenAI-class model capability but require it to run inside their existing Azure compliance/data-residency posture rather than calling OpenAI's API directly.
• AKS (Azure Kubernetes Service): Azure's managed Kubernetes — same role as EKS/GKE.
• Blob Storage: Azure's S3/GCS equivalent object storage.

Choosing Cloud-Neutral Patterns

Regardless of which cloud a stack runs on, the architecture patterns from the rest of this MLOps section — Containers, Kubernetes, CI/CD, Infrastructure as Code — are deliberately cloud-agnostic: a Docker container, a Kubernetes manifest, and a Terraform module all run on any of the three clouds above with minimal (often zero) changes, which is exactly why containerizing and using Kubernetes/Terraform rather than deeply coupling to any one cloud's proprietary managed services is the standard way to keep a real option to migrate or run multi-cloud, even for a team that has no near-term plan to actually do so.

Next: Deploying Models on AWS & Azure — the concrete methods (SageMaker, Azure ML, and the container-native alternatives) for actually getting a model running on this infrastructure.