• Define and evolve the end-to-end architecture for Unstructured’s data transformation and retrieval platform • Build and scale distributed systems that process massive volumes of unstructured data across diverse formats and sources • Serve as the authority on Kubernetes orchestration, cluster design, performance tuning, and reliability • Lead Python architecture and best practices • Design and optimize Postgres schemas, queries, and indexing strategies • Mentor senior engineers through design reviews • Partner with infrastructure and product teams to translate research prototypes into production-grade systems

• Design, provision, and manage Apache Kafka clusters (self-managed on GCP/AWS or via Confluent Platform / MSK). • Configure and tune brokers, ZooKeeper/KRaft, topics, partitions, replication factors, and retention policies for high throughput and low latency. • Perform cluster upgrades, rolling restarts, and broker replacements with zero downtime. • Implement and manage Kafka Connect pipelines for data ingestion and egress across heterogeneous systems. • Administer Kafka Streams and ksqlDB deployments for real-time stream processing workloads. • Maintain Schema Registry and enforce schema governance standards across teams. • Define and track SLIs/SLOs for consumer lag, throughput, end-to-end latency, and broker health. • Design and implement cloud infrastructure using IaC – Terraform • Build automated deployment pipelines for Kafka configuration changes using GitOps workflows (ArgoCD, Flux). • Create self-service tooling and runbooks to reduce toil for development teams. • Automate topic provisioning, ACL management, and schema registration via APIs and CLI tooling. • Integrate tools like GitLab CI/CD, or Cloud Build for automated testing and deployment. • Ensure seamless integration of data pipelines with other GCP services like Big Query, Cloud Storage. • Monitor and Optimize performance, reliability, and cost of Kafka and streaming pipelines • Implement security best practices for GCP resources, including IAM policies, encryption, and network security. • Ensure Observability is an integral part of the infrastructure platforms and provides adequate visibility about their health, utilization, and cost. • Collaborate extensively with cross functional teams to understand their requirements; educate them through documentation/trainings and improve the adoption of the platforms/tools.

• Design, develop, test, and maintain scalable front-end and back-end components using modern frameworks. • Architect and implement robust, secure APIs and data integrations. • Lead code reviews, mentor junior developers, and contribute to the continuous improvement of engineering practices. • Collaborate cross-functionally with product, design, and operations teams to define and ship high-impact features. • Participate in sprint planning, retrospectives, and other agile ceremonies.

• Lead the strategic extraction into a distributed architecture • Apply Conway's Law and utilize the Inverse Conway Maneuver • Guide the creation of the internal developer platform • Build frameworks for knowledge transfer and decision-making • Act as a translator between the C-suite and engineering teams • Manage technical debt and translate architectural constraints