Personal engineering journal
Writing
Focused engineering notes, product observations, and build logs.
10 posts
The Tenant Isolation Model: Building Multi-Tenant PostgreSQL Systems with tenant_id Scoping
An engineering deep dive into implementing multi-tenant data isolation using tenant_id-based row scoping in PostgreSQL, including design decisions, query patterns, indexing strategies, and the tradeoffs compared to separate schemas or dedicated databases.
Kafka DLQ + Circuit Breaker Design: Building a Fault-Tolerant Ingestion Pipeline in Upblit
Learn how Upblit designs a resilient Kafka ingestion pipeline using dead letter queues (DLQs), circuit breakers, exponential backoff retries, and failure recovery strategies to handle consumer crashes, database outages, and vector service downtime without losing data.
How Codily Implements Delayed Job Scheduling with Redis ZSETs
A deep technical dive into Codily's delayed job scheduling architecture using Redis Sorted Sets (ZSETs), Unix epoch nanosecond timestamps, and a scheduler goroutine that promotes ready jobs every two seconds with atomic `ZREM` + `RPUSH` operations and exponential backoff retries.
Idempotency Keys in Distributed Job Systems: Preventing Duplicate Work in Codily
A deep dive into why idempotency keys are essential for reliable asynchronous job systems, using Codily's `POST /jobs` `Idempotency-Key` header as a case study. Learn how request deduplication works, how duplicate submissions are handled, and why idempotency prevents costly double-processing in production systems.
Separating Product Secrets from Ops Secrets: Why Upblit Uses a Custom Secrets Manager Alongside HashiCorp Vault
An architectural deep dive into Upblit's decision to separate customer-facing secrets from infrastructure secrets by building a Lua-based Secrets Manager while reserving HashiCorp Vault for operational credentials. Learn why this separation improves security boundaries, scalability, and product design—and why combining both concerns is a common anti-pattern.
What I Learned Deploying 8 Services Across GCP, AWS, Azure, and Cloudflare as a Solo Developer
After deploying Upblit and Codily across multiple cloud providers as a solo developer, I realized that distributed systems are as much about networking, secrets management, and operational discipline as they are about writing code. This is a reflection on the challenges, mistakes, and lessons learned from running eight production services across GCP, AWS, Azure, and Cloudflare.
Job Dependencies in Codily: Building a Lightweight DAG Scheduler with `dependency_ids`
Explore how Codily implements job dependencies using `dependency_ids` to create a lightweight Directed Acyclic Graph (DAG) scheduler. Learn how dependent jobs wait for prerequisites, how cycles are prevented, and how this approach compares to workflow engines like Airflow and Temporal while remaining simple and efficient
Multi-Tenant RBAC with Header-Based Authentication: Evolving Codily from Dev Headers to OAuth Sessions
Explore how Codily implements multi-tenant role-based access control (RBAC) using `X-Tenant-ID` and `X-Tenant-Role` headers, the permission model behind `viewer`, `operator`, `admin`, and `owner` roles, and how the system evolved from development-friendly header authentication to OAuth-backed signed session cookies for production deployments.
Building a Go Worker Pool from Scratch: How Codily Processes Background Jobs
Learn how Codily implements a scalable worker pool in Go using configurable goroutines, Redis `BLPOP`, PostgreSQL job state management, and retry logic. This beginner-to-intermediate guide walks through the architecture and includes a simplified implementation of the core worker loop.
Designing Webhooks for Job State Changes: Why Codily Uses Fire-and-Forget Notifications
A deep dive into Codily's webhook architecture for notifying external systems when jobs reach terminal states. Learn about the webhook payload format, supported events, the 5-second timeout with no retry policy, and why a fire-and-forget approach is often the right tradeoff for asynchronous job processing systems.
Case Study
Long-form build breakdowns will live here.
Keep publishing regular posts now; this area is ready for deeper engineering case studies when you want to separate them from notes.