The Reality of Cloud Outages: Beyond Single-AZ Failures

Modern cloud providers offer 99.99% availability SLAs for availability zones, but region-level failures—triggered by fiber cuts, cascading control-plane panics, or DNS provider disruptions—remain inevitable. When an enterprise software platform serves global users, a regional outage can result in millions of dollars in downtime losses.

Transitioning from an Active-Passive disaster recovery setup to a true Multi-Region Active-Active architecture ensures that all regions serve read and write traffic simultaneously while guaranteeing data survivability across full regional blackouts.

Global Traffic Ingestion: BGP Anycast and Cloudflare Magic Transit

To achieve seamless active-active ingress, traffic cannot rely on GeoDNS alone due to client-side DNS caching and TTL delays. Instead, we utilize BGP Anycast routing. A single virtual IP is announced from hundreds of edge Points of Presence (PoPs) worldwide, automatically routing user requests to the nearest healthy cloud region over private backbone fiber.

Distributed Consensus at Scale: CockroachDB Multi-Region Tables

Traditional relational databases enforce write serialization through a single primary instance. In a multi-region setup, this creates massive cross-continental latency for write operations. CockroachDB solves this by partitioning data at the row level and distributing Raft consensus ranges across regions:

-- Define a tri-region database topology
ALTER DATABASE enterprise_saas SET PRIMARY REGION "aws-us-east-1";
ALTER DATABASE enterprise_saas ADD REGION "aws-eu-west-1";
ALTER DATABASE enterprise_saas ADD REGION "aws-ap-southeast-1";

-- Configure regional table for sub-5ms local reads and writes
CREATE TABLE customer_orders (
    order_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    region_id crdb_internal_region NOT NULL,
    customer_id UUID NOT NULL,
    total_amount NUMERIC(12, 2) NOT NULL,
    status VARCHAR(32) NOT NULL,
    created_at TIMESTAMPTZ DEFAULT clock_timestamp()
) LOCALITY REGIONAL BY ROW AS region_id;

-- Global reference table for cross-region ultra-fast reads
CREATE TABLE currency_exchange_rates (
    currency_code VARCHAR(3) PRIMARY KEY,
    rate_to_usd NUMERIC(10, 6) NOT NULL,
    updated_at TIMESTAMPTZ DEFAULT clock_timestamp()
) LOCALITY GLOBAL;

Survival Goals: Region Failure Simulation

With SURVIVE REGION FAILURE enabled, CockroachDB replicates data across at least three regions with a 5-way consensus replica factor. If an entire cloud region goes dark, the remaining two regions maintain a Raft quorum and elect new leaseholders within 4.5 seconds—without human intervention.