EngineeringSolution ArchitectData and Decisions

Session 01 - Cosmos DB & NoSQL Patterns

Cosmos DB Overview

Azure Cosmos DB là globally distributed, multi-model database với guarantees về latency, throughput, consistency và availability. Đây là lựa chọn hàng đầu cho workloads cần low latency và global scale.

Key Characteristics

FeatureValue
Latency< 10ms read, < 15ms write (P99)
Availability99.999% (multi-region)
ThroughputProvisioned RU/s hoặc Serverless
DistributionMulti-region, multi-write
APIsNoSQL, MongoDB, Cassandra, Gremlin, Table

Data Modeling - Embedding vs Referencing

Nguyên tắc quan trọng nhất

Model data theo cách app đọc, không phải theo cách relational database chuẩn hóa.

Khi nào Embed vs Reference?

CriteriaEmbed (Denormalize)Reference (Normalize)
Đọc cùng nhau?Luôn đọc chungĐọc riêng biệt
Tỷ lệ updateÍt updateUpdate thường xuyên
Cardinality1:few (< 100)1:many (unbounded)
Document size< 2 MB sau embedGần giới hạn 2 MB
ConsistencyAtomic (cùng document)Eventually consistent

Ví dụ: Order Document

json
// EMBEDDED - Order + OrderItems (1:few, always read together)
{
    "id": "order-123",
    "customerId": "cust-456",
    "customerName": "Nguyen Van A",
    "orderDate": "2024-01-15T10:30:00Z",
    "status": "confirmed",
    "items": [
        {
            "productId": "prod-789",
            "productName": "Laptop Dell XPS",
            "quantity": 1,
            "unitPrice": 25000000,
            "totalPrice": 25000000
        },
        {
            "productId": "prod-012",
            "productName": "USB-C Hub",
            "quantity": 2,
            "unitPrice": 500000,
            "totalPrice": 1000000
        }
    ],
    "totalAmount": 26000000,
    "partitionKey": "cust-456"
}

Partition Key Strategy

Chọn Partition Key - Decision Framework

Ví dụ Partition Key theo use case

Use CasePartition KeyLý do
E-commerce orderscustomerIdQuery by customer phổ biến nhất
IoT telemetrydeviceIdMỗi device ghi độc lập
Multi-tenant SaaStenantIdData isolation per tenant
Chat messagesconversationIdMessages luôn query theo conversation
Loggingdate + serviceIdTránh hot partition trên date alone

Consistency Levels

LevelGuaranteeLatencyRU CostUse Case
StrongLinearizable readsCao nhất2x RUFinancial transactions
Bounded StalenessReads lag ≤ K versions hoặc T timeCao2x RULeaderboards, inventory
SessionRead-your-writes trong sessionTrung bình1x RUDefault - hầu hết apps
Consistent PrefixReads không bao giờ out of orderThấp1x RUSocial media feeds
EventualKhông guarantee orderThấp nhất1x RUAnalytics, non-critical reads

Recommendation: Bắt đầu với Session consistency - đủ mạnh cho hầu hết scenarios, cost hợp lý.

RU Estimation

RU Cost Guidelines

OperationApproximate Cost
Point read (1 KB doc by id + PK)1 RU
Write (1 KB doc)~5 RU
Query (no index, scan)10-1000+ RU
Query (indexed, selective)3-10 RU

Estimation Formula

Required RU/s = (Reads/sec × RU per read) + (Writes/sec × RU per write)

Ví dụ:
- 1000 reads/sec × 1 RU = 1,000 RU
- 200 writes/sec × 5 RU = 1,000 RU
- Total: 2,000 RU/s
- Với buffer 20%: Provisioned = 2,400 RU/s

Change Feed Patterns

Event-Driven Architecture với Change Feed

csharp
// Change Feed Processor - reliable, scalable consumption
public class OrderChangeFeedProcessor : IHostedService
{
    private readonly Container _monitoredContainer;
    private readonly Container _leaseContainer;
    private ChangeFeedProcessor? _processor;

    public OrderChangeFeedProcessor(CosmosClient client)
    {
        var database = client.GetDatabase("OrderDb");
        _monitoredContainer = database.GetContainer("Orders");
        _leaseContainer = database.GetContainer("Leases");
    }

    public async Task StartAsync(CancellationToken ct)
    {
        _processor = _monitoredContainer
            .GetChangeFeedProcessorBuilder<OrderDocument>(
                processorName: "orderProcessor",
                onChangesDelegate: HandleChangesAsync)
            .WithInstanceName(Environment.MachineName)
            .WithLeaseContainer(_leaseContainer)
            .WithStartTime(DateTime.UtcNow.AddHours(-1)) // Catchup
            .WithMaxItems(100)
            .Build();

        await _processor.StartAsync();
    }

    private async Task HandleChangesAsync(
        ChangeFeedProcessorContext context,
        IReadOnlyCollection<OrderDocument> changes,
        CancellationToken ct)
    {
        foreach (var order in changes)
        {
            // Denormalize tới read model
            await UpdateReadModelAsync(order);

            // Publish domain event
            await PublishOrderEventAsync(order);
        }
    }

    public async Task StopAsync(CancellationToken ct)
    {
        if (_processor is not null)
            await _processor.StopAsync();
    }
}

Change Feed Use Cases

PatternMô tả
Materialized ViewsTạo read-optimized views từ write model
Event SourcingReplay events từ change feed
Real-time AnalyticsStream changes tới analytics pipeline
Cross-region SyncSync data tới other data stores
Cache InvalidationInvalidate Redis cache khi data thay đổi

Playbook: Setup Cosmos DB từ .NET

Azure CLI Setup

bash
# Tạo Cosmos DB account
az cosmosdb create \
    --name my-cosmos-account \
    --resource-group my-rg \
    --default-consistency-level Session \
    --locations regionName=southeastasia isZoneRedundant=true \
    --enable-automatic-failover true

# Tạo database
az cosmosdb sql database create \
    --account-name my-cosmos-account \
    --resource-group my-rg \
    --name OrderDb \
    --throughput 400

# Tạo container với partition key
az cosmosdb sql container create \
    --account-name my-cosmos-account \
    --resource-group my-rg \
    --database-name OrderDb \
    --name Orders \
    --partition-key-path /customerId \
    --throughput 400

.NET Integration

csharp
// Program.cs
builder.Services.AddSingleton(sp =>
{
    var connectionString = builder.Configuration["CosmosDb:ConnectionString"];
    return new CosmosClient(connectionString, new CosmosClientOptions
    {
        SerializerOptions = new CosmosSerializationOptions
        {
            PropertyNamingPolicy = CosmosPropertyNamingPolicy.CamelCase
        },
        ConnectionMode = ConnectionMode.Direct, // Production: Direct mode
        MaxRetryAttemptsOnRateLimitedRequests = 5,
        MaxRetryWaitTimeOnRateLimitedRequests = TimeSpan.FromSeconds(30)
    });
});

// Repository pattern
public class OrderRepository
{
    private readonly Container _container;

    public OrderRepository(CosmosClient client)
    {
        _container = client.GetContainer("OrderDb", "Orders");
    }

    public async Task<Order> GetAsync(string orderId, string customerId)
    {
        var response = await _container.ReadItemAsync<Order>(
            orderId, new PartitionKey(customerId));
        return response.Resource;
    }

    public async Task<IEnumerable<Order>> GetByCustomerAsync(string customerId)
    {
        var query = new QueryDefinition(
            "SELECT * FROM c WHERE c.customerId = @customerId ORDER BY c.orderDate DESC")
            .WithParameter("@customerId", customerId);

        var results = new List<Order>();
        using var iterator = _container.GetItemQueryIterator<Order>(
            query, requestOptions: new QueryRequestOptions
            {
                PartitionKey = new PartitionKey(customerId),
                MaxItemCount = 50
            });

        while (iterator.HasMoreResults)
        {
            var response = await iterator.ReadNextAsync();
            results.AddRange(response);
        }

        return results;
    }
}

Trade-offs

Quyết địnhProCon
Cosmos DB vs SQL ServerGlobal scale, low latencyCost cao, no JOINs, learning curve
Embed everythingFast reads, atomic writesDocument bloat, update complexity
Reference like relationalFlexible, normalizedMultiple queries, no transactions across docs
Strong consistencyNo stale reads2x RU cost, higher latency
Session consistencyGood balance, affordableStale reads across sessions
Serverless vs ProvisionedPay per use, no idle costCold start, RU limit per request