EKS migration
Moving to Kubernetes is not a lift-and-shift. The application's assumptions — local state, startup order, direct host access — become the migration's actual work.
Typical starting point
An on-premises or self-managed container runtime (Docker Swarm, standalone Docker, or a hand-built Kubernetes cluster) running microservices with:
- Manual scaling limited by available hardware.
- Operational overhead absorbed by a small team.
- Limited integration with managed AWS services.
- Deployment processes that depend on the infrastructure in front of them.
Migration phases
Assessment and planning Resource-utilisation analysis, dependency mapping, and a security and compliance review of the target design.
Architecture design Cluster configuration, node-group strategy, network design, and CI/CD integration.
Migration execution Registry migration, manifest conversion, data migration, and cutover.
Automation and monitoring Infrastructure as code, autoscaling, logging, tracing, and alerting.
Testing and validation Functional, performance and security testing against production-shaped load.
Training and handover Workshops for the operating team, runbooks, and an on-call model.
1. Assessment
| Question | Why it matters |
|---|---|
| What is the CPU and memory profile per service? | Determines node group shape and requests/limits |
| Which services hold local state? | Those need a persistent volume or an externalisation plan |
| What is the startup dependency order? | Kubernetes does not enforce ordering; readiness probes and init containers do |
| Which services need host-level access? | HostPath, privileged containers and Docker socket access do not transfer cleanly |
| What is the inter-service communication pattern? | Determines service discovery and network policy design |
| What is the deployment cadence per service? | Determines repository and pipeline structure |
| What data stores are involved? | Determines whether migration is a copy, a replication, or a re-platform |
| What compliance obligations apply? | Determines logging, encryption and network isolation requirements |
2. Architecture
Route 53
│
▼
ALB (AWS Load Balancer Controller)
│
▼
EKS cluster
├── node group: general (on-demand, 3 AZ)
├── node group: batch (spot, tainted)
└── Fargate profile: isolated workloads
│
├── namespace: app
├── namespace: platform
└── namespace: observability
Decisions to record:
- Cluster topology — one cluster per environment; one per region for resilience.
- Node strategy — managed node groups with mixed instance types; spot for interruption-tolerant workloads.
- Ingress — AWS Load Balancer Controller, one ALB per environment, host- and path-based routing.
- Identity — IRSA for pod-level AWS access; no node instance role used by application pods.
- Network — VPC CNI with network policies; private subnets for nodes.
- Storage — EBS CSI for block, EFS CSI for shared file, S3 for objects.
3. Migration
Images
- Move images to ECR. Rebuild rather than copy where the original image has an unmaintained base.
- Re-scan; the migration is a good moment to fix accumulated CVEs.
- Pin digests in manifests.
Manifests
| Source | Target |
|---|---|
docker-compose.yml | Deployment + Service + ConfigMap + Secret |
| Environment variables | ConfigMap (non-secret) and Secret / External Secrets |
| Volume mounts | PersistentVolumeClaim or emptyDir |
| Port publishing | Service + Ingress |
| Restart policy | Deployment strategy and probes |
| Depends-on | Init containers and readiness probes |
Helm or Kustomize for templating. Plain manifests duplicated per environment diverge quickly.
Data
| Store | Approach |
|---|---|
| PostgreSQL / MySQL | AWS DMS replication with a short cutover, or dump and restore with a maintenance window |
| Object storage | aws s3 sync or DataSync |
| Cache | Rebuild; do not migrate cache contents |
| Search index | Reindex from source rather than migrate the index |
Data migration has the least reversible failure mode. Rehearse it, time it, and keep the source available until the target is verified.
Cutover
1. Deploy the target alongside the source
2. Replicate data continuously
3. Run the target in shadow or read-only mode
4. Verify against production traffic
5. Freeze writes on source
6. Final replication catch-up
7. Switch DNS / traffic
8. Monitor, then decommission source
4. Automation and monitoring
- Infrastructure — Terraform for the cluster, node groups, IAM, and add-ons.
- Application delivery — GitOps (Argo CD or Flux) or the CI pipeline applying manifests; one owner per path.
- Autoscaling — HPA on concurrency or custom metrics; Karpenter or cluster autoscaler for nodes.
- Logging — Fluent Bit or CloudWatch agent to CloudWatch Logs, structured.
- Metrics — Container Insights or Prometheus with the RED method per service.
- Tracing — OpenTelemetry or X-Ray.
- Alarms — error rate, latency, pod restarts, pending pods, node pressure, certificate expiry.
5. Validation
| Test | Criterion |
|---|---|
| Functional | Every service behaves as before, verified by contract or end-to-end tests |
| Performance | Load test at expected peak and 1.5× peak; p95 within target |
| Resilience | Kill a node; the workload reschedules without user-visible failure |
| Security | Image scan clean; no privileged pods; network policies effective |
| Backup and restore | Restore a namespace and its data into a clean cluster |
| Rolling update | Deploy a new version with zero failed requests |
6. Handover
- Architecture and decision record.
- Runbooks for the operational tasks the team will actually perform.
- A game day: induce a node failure, a bad deployment and a certificate expiry in a non-production cluster, and fix them.
- An on-call model with escalation paths.