Architecting high-throughput streaming, CDC & analytical platforms at enterprise scale.
Lead Software Engineer and Distributed Systems Architect with nearly 8 years of experience across LeadSquared and Hewlett Packard Enterprise (HPE). Core architect and major code contributor (3,680+ commits across a 29,000+ commit Go monorepo) for a multi-tenant cloud platform serving ~2,000 enterprise customers and processing ~2M+ daily Kafka and Debezium CDC streaming events.
Windowed batch accumulation & in-memory deduplication for ClickHouse writes
Event-driven Kubernetes & KEDA consumer autoscaling with scale-to-zero
Production memory leak elimination (65%) & storage schema optimization (70%)
~2M daily Kafka & CDC events processed across ~1,000 active streaming tenants
Core Architectural Transformations
Platform-wide engineering initiatives built for high availability, strict data consistency, and cloud cost efficiency.
Windowed Kafka-to-ClickHouse Batching Engine
Eliminated synchronous single-record database round-trip bottlenecks across ~2M daily Kafka events by designing a windowed batch-accumulation and in-memory entity deduplication layer. Enforced correctness invariants and flush-barrier synchronization across interleaved schema and data streams.
Event-Driven Kubernetes & KEDA Autoscaling
Re-architected Kubernetes consumer orchestration using KEDA and AWS MSK, simplifying cluster topology by replacing a fragile custom master-worker pod model with automated scale-to-zero policies, level-triggered self-healing reconciliation, and per-workload resource tuning.
Distributed SQL Query Engine & Dynamic Pivot Compiler
Scaled and stabilized the analytical query compiler powering cross-dataset Common Table Expression (CTE) joins, multi-dimensional pivot tables, and runtime-parametric filter evaluation across Abstract Syntax Trees (ASTs) with full backward compatibility.
Pipeline Fault Isolation & Multi-Team Integration
Deprecated fragile delay-queue retries in favor of deterministic infrastructure-vs-payload error classification, Dead-Letter Queue (DLQ) routing, and strict schema safety guardrails. Acted as central technical authority across 5 external data-producing engineering teams.
Re-Architecting Multi-Tenant Streaming Ingestion & KEDA Consumer Orchestration
1. Architectural Decisions & Trade-Offs
Replaced synchronous per-message ClickHouse writes (~80% single-record payloads) with a windowed batch-accumulation and within-window deduplication engine. Because partitions interleave schema mutations and entity data, per-partition worker pool parallelism was explicitly ruled out to prevent stale metadata reads—enforcing 8 formal correctness invariants and flush-barrier semantics instead.
On the Kubernetes consumer controller, eliminated our custom master/child pod hybrid entirely by leveraging KEDA's native minReplicaCount: 1 baseline alongside partition-lag scaling, reducing cloud compute costs by 40%.
2. Retrospective & Next-Gen Evolution
Challenging Legacy Topologies Early: Rather than initially building KEDA around our legacy master/child pod model before deleting the master layer later, challenging that legacy invariant during the initial design phase would have saved significant iteration cycles.
Randomized Jitter on Back-Pressure Retries: While segregating transient infrastructure faults from payload errors enabled clean back-pressure retries, deterministic backoff intervals can cause concurrent consumers to retry in lockstep during database recovery. Adding randomized jitter (noise) to infrastructure retry backoffs de-correlates concurrent query spikes and prevents thundering-herd load.
Professional Experience
Lead Software Engineer — LeadSquared
Apr 2025 – PresentCore architect for a multi-tenant analytics platform (~2,000 enterprise tenants, ~2M daily Kafka events), delivering a ~100× reduction in ingestion lag, building real-time Debezium binlog CDC pipelines into Kafka, and cutting cloud compute spend by 40%. Central technical lead across 5 partner engineering teams, driving architecture specifications, production readiness, code review governance, and engineering mentorship.
Senior Software Engineer — LeadSquared
Apr 2024 – Mar 2025Owned the dynamic pivot-table engine and stabilized multi-dataset CTE joins across the ClickHouse SQL compiler. Eliminated critical memory leaks to reduce memory footprint by 65%, optimized configuration storage to cut storage costs by 70%, and resolved complex UTC timezone scheduling defects.
Software Engineer — LeadSquared
Dec 2021 – Mar 2024Engineered customer-facing analytical REST APIs, incremental user-sync ETL pipelines, audit trails, and automated migration CLI utilities that transitioned production customers from legacy reporting models to a modern multi-tenant architecture.
Software Engineer & Intern — Hewlett Packard Enterprise (HPE)
Jan 2019 – Dec 2021Built Go, Java, Python, GraphQL, Kafka, and Apache Druid microservices for HPE Storage Cloud Data Services, shipping the core REST API framework that migrated the platform from on-premise hardware to the cloud. Won 1st Prize in the HPE Storage Hackathon for an automated L1/L2 support chatbot projected to save $100 per query, and developed ML fault-prediction models and OS provisioning automation for HPE Synergy servers.
PES University, Bangalore (2015 – 2019)
B.Tech in Computer Science & Engineering (Data Science Specialization) — GPA: 9.55 / 10.0
Multi-semester recipient of the CNR Rao Merit Scholarship. Designed and delivered a hands-on Big Data with Python course to undergraduate students as Subject Matter Expert (SME) for PESU I/O.
ISPRS-GEOGLAM-ISRS Conference Publication
Co-authored and presented 'Deep Learning and Statistical Models for Detection of White Stem Borer Disease in Arabica Coffee' (Feb 2019) in joint research with the Indian Space Research Organisation (RRSC-ISRO) and Coffee Board of India. Previously served as Research Intern at IIT Mandi (2017 & 2018) modeling network failure prediction using LSTM, NARX, and ARIMA architectures.