Greystar Real Estate Partners, also known by names like Greystar Worldwide and Greystar Apartments, is an international real estate development and management f
DataOps Engineer
Location
United States
Posted
10 days ago
Salary
$120K - $150K / year
Seniority
Mid Level
Job Description
DataOps Engineer
Greystar Real Estate Partners
Role Description Greystar is seeking a DataOps Engineer to join the Data Marketplace (DMP) team. This is a deeply technical, hands-on platform engineering role at the core of Greystar’s enterprise data infrastructure — a Databricks-native medallion architecture (Bronze → Silver → Gold) running entirely on Microsoft Azure. You will own the reliability, scalability, and operational excellence of the DMP platform, working within the DataOps pod inside the broader Analytics Engineering umbrella. This role is Databricks and Azure-heavy. Most of your day lives inside Databricks — Delta Live Tables, Unity Catalog, Jobs, Workflows — backed by the full Azure data services stack including ADF, ADLS Gen2, Azure Monitor, Key Vault, and more. Deep mastery of both platforms is a baseline expectation, not a differentiator. Critically, we expect this engineer to use AI as a first-class tool in their DataOps and observability practice — today, not eventually. That means AI-driven pipeline diagnostics, LLM-assisted root cause analysis, intelligent anomaly detection, and agentic observability agents that surface issues before they reach production. If you are still approaching DataOps the same way you did three years ago, this is not the right role. We are building self-aware, self-healing data infrastructure and need an engineer who is already operating that way. You will also own the full deployment lifecycle — promoting data pipeline changes and platform configurations across dev, staging, and production environments using GitHub Enterprise and Linear for structured release management. Strong CI/CD discipline, environment promotion hygiene, and release coordination are as important here as pipeline engineering craft. Qualifications - 7+ years of DataOps, data engineering, or platform engineering experience in a production environment - Expert-level hands-on experience with Databricks: Delta Live Tables, Jobs/Workflows, Unity Catalog, Spark performance tuning, and Delta Lake internals - Strong command of the Azure data services ecosystem: ADF, ADLS Gen2, Azure Monitor, Log Analytics, Key Vault, and related services - Demonstrated, production use of AI tools in DataOps or data observability workflows — LLM-assisted diagnostics, intelligent alerting, agentic monitoring, or equivalent - Proven CI/CD experience using GitHub Enterprise — branch strategies, PR automation, environment promotion, and release management for data pipelines - Solid Python and/or Scala skills for pipeline development; SQL fluency for Gold layer transformation and DQ validation - Hands-on experience with ADF pipeline design and orchestration at scale - Experience with medallion / lakehouse architecture patterns and multi-environment deployment discipline - Strong collaborative skills across engineering, governance, and business stakeholder teams Requirements - Implement AI-powered observability — using LLMs and ML models to detect pipeline drift, classify anomalies, predict SLA risk, and generate automated incident summaries - Build agentic monitoring workflows that proactively surface data quality degradation, pipeline dropout, schema drift, and volume anomalies across all DMP layers - Integrate AI tooling (Databricks Mosaic AI, Genie, OpenAI APIs, or equivalent) into operational DataOps processes — not as experiments, but as production-grade capabilities - Develop and maintain AI-assisted root cause analysis tooling to reduce MTTR on pipeline failures, with structured learnings fed back into the platform - Contribute to Greystar’s 18-month agentic AI roadmap, leading near-term delivery of self-healing pipeline capabilities - Operate the full Azure data services stack supporting DMP: ADLS Gen2, Azure Data Factory (ADF), Azure Monitor, Log Analytics, Key Vault, and Event Hub - Design and maintain ADF pipelines for source system ingestion, including orchestration patterns for multi-tenant ERP environments (Yardi, Entrata, RealPage) - Collaborate with Azure infrastructure and cloud engineering teams on networking, identity, security, and resource provisioning - Drive cost governance through Azure Cost Management, Databricks DBU optimization, and storage lifecycle policies - Own the design, build, and optimization of data pipelines on Databricks using Delta Live Tables (DLT), PySpark, Workflows, and Jobs across the full DMP medallion stack - Administer and govern the Databricks workspace: Unity Catalog, cluster policies, access controls, compute configurations, and Delta table lifecycle management - Tune Spark jobs for performance, reliability, and cost — profiling bottlenecks, optimizing partitioning, managing Z-ordering, and controlling compute spend - Leverage Databricks Mosaic AI and Genie to build AI-native DataOps capabilities including intelligent pipeline monitoring, anomaly detection, and natural language data access - Architect and enforce DMP platform standards: naming conventions, schema evolution policies, SLA tiers, and medallion layer contracts - Own the full deployment pipeline for DMP data workflows — promoting changes from development through staging to production with rigor and minimal disruption - Build and maintain CI/CD workflows using GitHub Enterprise, including branch strategies, pull request automation, environment-specific configuration management, and release gating - Use Linear for sprint planning, release tracking, and issue management across deployment cycles; coordinate engineering work items with cross-functional stakeholders - Enforce deployment standards: automated testing gates, rollback procedures, change documentation, and environment parity controls - Partner with the analytics engineering and integration teams to align deployment cadences across the DMP stack - Instrument DQ checks across Bronze, Silver, and Gold layers covering completeness, consistency, accuracy, uniqueness, and referential integrity - Partner with Brett Finley’s Data Governance team to enforce data contracts, ownership standards, and quality SLAs within Unity Catalog - Build feedback loops between DQ scoring, pipeline observability, and upstream source owners to drive systemic data reliability improvements - Partner with analytics engineers, data governance, and product stakeholders to align pipeline and platform design with business requirements - Produce thorough technical documentation — runbooks, deployment playbooks, incident post-mortems, ADRs, and platform specs - Participate in on-call rotation and support SLA commitments for business-critical DMP data domains Benefits - A high-impact role at the center of Greystar’s enterprise data transformation - Collaborative, engineering-driven team culture with a strong focus on craft, automation, and continuous improvement - Access to cutting-edge tooling — Databricks, full Azure stack, GitHub Enterprise, and an active AI innovation agenda - Competitive compensation, comprehensive benefits, and flexible work arrangements - Opportunity to define the DataOps discipline and lead Greystar’s self-healing pipeline and agentic AI roadmap - The salary range for this position is $120,000 - $150,000 USD Annually.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer
WriskWrisk provides digital insurance solutions, specializing in automotive solutions, to create personalized and accessible insurance options through innovative technology. Wrisk promo
Senior Data Engineer Remote Full time Edinburgh, Scotland, United Kingdom Description As a Senior Data Engineer with Atto, you will be part of our Data Science & Engineering team, helping our customers use transactional intelligence to drive smarter financial decisions and build impactful products. You’ll play a key role in designing and delivering scalable, production-ready analytical solutions that turn complex data into meaningful insights. You will own and improve the systems that ingest, transform, and serve large-scale transactional data sets. You will be responsible for designing resilient data pipelines, implementing robust data models, deploying real-time and batch processing systems, and introducing observability, testing and automation to keep our data trustworthy at scale. You will work closely with the data science and product teams to understand customer needs and the direction of internal strategy, producing measurable data engineering outcomes. We’re looking for someone who’s passionate about technology, curious about data infrastructure, and excited to solve real problems. You’ll need a strong foundation in data engineering, a creative mindset, and the ability to communicate clearly with both technical and non-technical stakeholders. You’ll be supported with training and development opportunities to deepen your expertise and stay ahead of industry trends. Responsibilities - Design, build, and operate data ingestion and ETL/ELT pipelines for our data systems that power Atto products and customer-facing applications. - Build and maintain our data platform architecture and infrastructure, including data pipelines, warehouses that power analytics and product features and reporting. - Implement data modelling, schema design, and data contracts to make datasets easy to consume. - Introduce and maintain observability, data quality checks, and automated testing for our production data flows. - Driving impact at scale by improving data workflows, service reliability, and making Atto’s capabilities faster, more reliable and widely accessible. - Collaborate with product managers and cross-functional teams to translate market signals and customers’ needs into innovative data-driven solutions. - Create new systems and approaches, while continuously improving and scaling existing ones for performance, reliability, and efficiency. - Conduct deep analysis of transactional datasets to validate data models, surface data quality issues, and propose engineering fixes. - Optimise performance and cost of data processing and storage across cloud services. - Communicate technical findings and recommendations clearly to both technical and non-technical stakeholders across the business. - Stay current with industry developments, emerging technologies, and best practices in data engineering. Requirements - A Bachelor's degree in Computer Science, Data/Software Engineering, or a related discipline. - 5+ years of hands-on experience in data engineering with a proven track record of delivering scalable, production-grade data platforms and data models. - Strong SQL skills and proven experience designing, building and optimising data pipelines and data warehouse schemas. - Strong proficiency in Python for production data engineering with a focus on building maintainable, testable and clean code. - Track record of implementing data quality, observability, monitoring and testing practices. - Solid experience with at least one cloud provider (Azure, AWS or GCP), and their managed data services. - Experience with CI/CD, infrastructure-as-code and working in containerised environments. - Familiarity with modern data stack tools: Databricks, Snowflake or RedShift, etc. - Experience working with reporting and modelling tools such as Power BI, including designing performant data models and supporting self-serve analytics. - Advanced experience in pattern recognition with the ability to translate complex data into actionable insights. - A results-oriented mindset, able to take concepts from ideation through to tangible outputs. - Comfortable working in a fast-paced, delivery-focused environment, balancing multiple priorities with agility. - A proactive and collaborative approach, with a willingness to iterate, share knowledge, and challenge assumptions constructively Bonus Points (We’re getting greedy) - Experience building or scaling data platforms in large-scale, regulated data environments. - Experience with DevOps for data, platform engineering, or building large data pipelines and real-time data streaming systems. - Exposure to working in a high-growth, evolving environment where the customer is at the heart of product and engineering decisions. . Benefits - A team of passionate, interesting people committed to your development and success - £70-£80k gross/pension - Personal training and Continuous Professional Development budget (CPD) - Uncapped bike to work scheme - Half-day Fridays every last week of the month to recharge - Wellness partnerships - In-person and virtual team events and workshops - Volunteering (Social Good Connect partnership) - 33 days holiday allowance to take when you want - £200 home working contribution to make sure you have everything you need to do your best work (get comfy - we want you to stay) - Ask us about our remote-first, flexible culture - this is core to who we are, and we're rated one of Scotland's most flexible employers Creating a more predictable future for lenders We are on a mission to enable our customers across the globe to effortlessly make use of real-time transaction data to better understand their customers, grow their business, revolutionise their offerings and delight with customer service. At Atto, you will be working for a business that is creating a more predictable future for lenders through our real-time transaction data platform. We use today's data to better predict tomorrow. This is an exciting stage in our growth, and we'd love you to be part of the story. Don’t speculate. Calculate.
• Migrar y ordenar código suelto: Tomar las queries SQL y scripts informales creados por los equipos de análisis y traducirlos a modelos limpios, estructurados y automatizados dentro de Dataform. • Armar y automatizar los "Bundles": Empaquetar el código de los pipelines junto con sus configuraciones para que se desplieguen de forma automática a producción usando GitLab CI/CD y Terraform, eliminando los pasos manuales. • Validar los datos codo a codo con el negocio: Trabajar directamente con los usuarios que usan los datos para entender sus necesidades, resolver dudas y asegurar juntos que las nuevas tablas reflejen la lógica real del negocio. • Acelerar el desarrollo usando IA (Claude/GPT): Utilizar asistentes de Inteligencia Artificial mediante Prompt Engineering para optimizar queries lentas, documentar el código de forma rápida y generar pruebas de calidad automáticas. • Optimizar el rendimiento en BigQuery: Aplicar técnicas de particionado y clustering en las tablas para asegurar que las consultas de los usuarios sean rápidas, eficientes y no generen costos innecesarios en la nube.
Data Engineer – Analytics, Modeling
In All MediaImagine the future of business. Ideas for a Digital Renaissance.
• Provide critical technical execution and analytical leadership, acting as a driving force in translating raw data into robust, production-ready data models. • Ensure cross-functional teams have seamless access to un-compromised, highly performant, and real-time datasets. • Responsible for dismantling legacy data workflows, engineering scalable data pipelines, and establishing rigorous validation standards to guarantee data reliability and pipeline health.
• Create and maintain complex, enterprise-scale data pipelines and foundational datasets while defining technical strategy and architectural direction for advertising products • Design and build sophisticated ETL processes, data models, and analytical frameworks using SQL, Python, and modern data stack technologies • Build and maintain the data infrastructure that powers Ads ML - feature pipelines, label generation workflows, and training data systems that enable our ranking and delivery models • Develop data quality frameworks, monitoring systems, automated anomaly detection, and alerting infrastructure that operates at massive scale • Collaborate with data scientists, ML engineers, and product teams to identify high-impact data infrastructure opportunities, owning design through implementation • Drive cross-functional technical initiatives solving sophisticated data engineering challenges • Build scalable rubrics that help lead and mentor engineers through projects that accelerate launch velocity and harden data systems • Navigate ambiguity and make sound technical decisions with incomplete information, balancing short-term delivery with long-term infrastructure investment



