Digest #151: Uber’s Migration, Terraform Tips, AMI Creation, and Helm Chart Scanning

Learn zero-downtime migration techniques, improve testing workflows, and master AMI creation. Plus, explore Terraform tools, Helm chart validation, and debugging AWS EC2 issues.

Welcome to this week’s edition of the DevOps Bulletin!

This week, we cover how teams solve big challenges. Learn about Uber’s zero-downtime migration for billions of trips, Amazon DynamoDB’s price drop, and the challenges of managing networks in multi-cloud setups. We also look at the risks of using Terraform automation tools and a new Amazon S3 feature to append data to objects.

Our tutorials focus on useful, hands-on lessons. Learn how to test Go applications with PostgreSQL, create AMIs with Image Builder and Terraform, and set up serverless REST APIs using AWS API Gateway, Lambda, and DynamoDB. Discover how Mercari planned a migration to a new system without downtime and how to debug SSH connection issues in AWS EC2.

This week’s open-source projects bring helpful tools for your work. Nova checks Helm charts in Kubernetes clusters. Pglite-Fusion embeds SQLite into PostgreSQL for easier multitenancy. Goliat helps manage and optimize Terraform deployments. pv-migrate makes moving Kubernetes PersistentVolumeClaim data simple.

All this and more in this week’s DevOps Bulletin—don’t miss out!

Newsworthy Stories

• Building databases over a weekend

• Migrating billions of DNS records

• Amazon S3 now supports appending data to an object

• The dangers of Terraform automation platforms

• FOCUS 1.1 is now available

• Hyrum's law in Golang

• Amazon DynamoDB reduces prices

• Cloud dev environments

Multi-Cloud Network Observability

This episode discusses network observability, the evolution of SRE for networking, flow analysis, and the complexity and tradeoffs of network design and operations at scale.

Tutorials of the week

• Testing with Go and PostgreSQL: Learn how to use ephemeral PostgreSQL instances in Go to simplify tests, reduce resource usage, and improve CI efficiency. Practical tips included!

• Creating an AMI with Image Builder: Learn to use Packer and Terraform to create pipelines, manage custom AMIs, and streamline automation.

• Deploying a Serverless REST API: Learn how to deploy a REST API with AWS API Gateway, Lambda, DynamoDB, and Cognito using Terraform.

• Designing a Zero Downtime Migration: Discover how Mercari plans a seamless migration to a double-entry bookkeeping system with Go, Kubernetes, and Cloud Spanner.

• Complete guide on Docker: Learn to build and deploy your distributed applications easily to the cloud with Docker.

• I have asked this SSH question in every AWS interview: Learn the best practices for debugging SSH connection issues in AWS EC2, focusing on security groups, network configurations, and system-level settings.

• Uber’s billion trips migration setup: Learn about their backward compatibility layer, shadow validation, traffic pinning, and real-time observability for seamless transitions.

Projects of the week

Highlighting cool DevOps projects to keep an eye on:

• Nova scans your cluster for installed Helm charts and then cross-checks them against all known Helm repositories.

• Pglite-fusion allows you to embed an SQLite database in your PostgreSQL table. AKA multitenancy has been solved.

• Goliat is an open-source tool for managing, visualizing, and optimizing Terraform deployments, with integration into Terraform Cloud and a custom provider.

• pv-migrate is a CLI tool and a kubectl plugin to easily migrate the contents of one Kubernetes PersistentVolumeClaim to another.