Anything from anywhere in your city in minutes.
Data Engineer II
Location
India
Posted
2 days ago
Salary
0
Seniority
Senior
Job Description
Data Engineer II
MRSOOL | مرسول
**What You Will Do ❓** - Design, build, and maintain scalable batch and real-time data pipelines using Maxwell, Kafka, Spark, and dbt to power analytics and business-critical applications. - Develop and optimize data models following Medallion Architecture (Bronze, Silver, Gold) to create reliable, reusable, and high-quality datasets. - Build and maintain cloud-native data platforms using S3, Spark, Trino, and BigQuery, ensuring scalability, reliability, and cost efficiency. - Design robust data ingestion frameworks leveraging CDC (Maxwell), Kafka, and event-driven architectures to support near real-time data processing. - Create, optimize, and maintain data warehouses and data marts that enable fast, reliable reporting and self-service analytics. - Partner closely with Product Managers, Data Analysts, Backend Engineers, and Business stakeholders to translate business requirements into scalable data solutions. - Develop reusable dbt models, testing frameworks, and documentation to improve data quality, governance, and developer productivity. - Optimize Spark jobs, Trino queries, and storage layouts for performance, reliability, and cost efficiency. - Own the end-to-end lifecycle of critical data pipelines, ensuring high availability, monitoring, SLA adherence, and proactive incident resolution. - Build and enhance the core data platform by developing reusable frameworks, automation, CI/CD pipelines, and engineering best practices. - Ensure data quality through validation, monitoring, lineage, and observability while implementing best practices for security and governance. - Enable analytics teams by delivering trusted datasets, semantic models, and dashboards that power decision-making through Metabase.
Job Requirements
- What We're Looking For 🚀**
- 4+ years of hands-on experience designing and building scalable data platforms, data lakes, and data warehouses.
- Strong proficiency in Spark (Scala, python) and SQL, with experience building production-grade data pipelines and distributed data processing applications.
- Hands-on experience with Apache Spark and a solid understanding of distributed data processing, performance tuning, and optimization.
- Experience building batch and streaming data pipelines using technologies such as Kafka, CDC/Maxwell, or similar event-driven architectures.
- Strong understanding of modern data lake architectures, including Medallion Architecture, data modeling, partitioning, and storage optimization.
- Experience working with cloud-native data platforms and technologies such as Amazon S3, BigQuery, Trino, or similar analytics engines.
- Solid experience designing dimensional models, star schemas, and building reliable data marts that support analytics and business intelligence.
- Hands-on experience with dbt, including developing reusable models, implementing automated testing, and maintaining documentation.
- Strong knowledge of data quality, observability, lineage, and engineering best practices to build reliable and maintainable data products.
- Experience optimizing large-scale data pipelines, SQL queries, and distributed processing jobs for performance, scalability, and cost efficiency.
- Familiarity with CI/CD, Git-based development workflows, infrastructure automation, and modern software engineering best practices.
- Excellent problem-solving skills with the ability to independently own projects from design through production.
- Strong communication and stakeholder management skills, with experience collaborating across Product, Engineering, Analytics, and Business teams.
- A passion for building scalable data platforms and continuously improving developer experience, platform reliability, and operational excellence.
Benefits
- What We Offer You❗**Inclusive and Diverse Environment: We foster an inclusive and diverse workplace that values innovation and offers remote environments.
- Competitive Compensation: Our compensation packages are highly competitive and include potential share options for certain roles.
- Personal Growth and Development: We are committed to your personal and professional growth, providing regular training and an annual learning stipend to help you advance your career in a dynamic environment.
- Autonomy and Mentorship: You'll enjoy a high degree of autonomy in your role, supported by mentorship and ambitious goals that pave the way for both your success and the company's growth.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Role Description We're looking for a Finance Engineer — someone who can turn manual finance and accounting processes into scalable, automated, AI-supported workflows. This role sits at the intersection of Finance, Accounting, Systems, Automation, and AI. - Build and improve workflows for reporting, forecasting, month-end review, reconciliations, and financial controls. - Work directly with clients: understand their finance and accounting needs, configure tools, support vendor onboarding, and own system improvements end-to-end. - Translate finance and accounting needs into system logic, automations, workflows, and documentation. - Automate repetitive finance work using AI tools, low-code tools, integrations, and internal workflows. - Connect and structure data from accounting systems, CRMs, payment tools, spreadsheets, BI tools, and data warehouses. - Build checks, controls, and validation logic to improve accuracy and reduce manual errors. - Partner with our engineering team on APIs, integrations, data pipelines, and workflow orchestration. Qualifications - 5+ years of experience in finance systems, finance operations, automation, or engineering roles within complex multi-stakeholder or enterprise environments. - 1+ year of hands-on experience building AI-powered automations, agents, prompts, workflows, or internal tools (Claude / Claude Code, Cursor, Copilot, custom LLM builds, or similar). - Experience with APIs, system integrations, data warehouses, and automation tools. - Strong understanding of finance and accounting processes: month-end close, reconciliations, AP / procure-to-pay, revenue, reporting, and corporate finance. - Ability to understand how financial data flows between departments and systems. - Fluent English (C1+) and excellent client-facing communication skills — you'll work directly with customers' founders and finance teams. - Strong drive to eliminate repetitive manual work. - Proactivity and high level of responsibility. - Ability to work in a fast-paced environment. Requirements - Python or other scripting experience. - Experience administering financial, accounting, billing, or ERP systems. - Experience with tools like NetSuite, QuickBooks, Xero, Stripe, Salesforce, HubSpot, Zuora, BigQuery, Snowflake, PostgreSQL, Looker, Power BI, Tableau, Zapier, Make, n8n, or Workato. - Proven impact in numbers: hours saved, manual work reduced, faster reporting, or fewer errors. - Experience working with US-based clients or startups. Recruitment Process - Soft skills interview with the recruiter. - Hard skills interview with the Head of FP&A. - Test task. - Culture fit interview with the VP of Operations. Benefits - Build the future of finance work — design AI-native finance operations, not just maintain them. - Rapid skills improvement (you'll work with multiple client businesses with different models and challenges). - Growth opportunities according to our seniority grading (it comes with a compensation increase). - Flexible schedule and time-off policy. - 18 days of paid vacation per year, paid sick leaves.
Data Engineer GCP (Remote)
bet365At bet365, we're one of the world's leading online gambling companies, revolutionising the industry since 2000. Founded by Denise Coates CBE, we now employ over 9,000 people and serve over 100 million customers in 27 languages. Focus on In-Play betting has solidified our market-leading position. Offering an unmatched experience across 96 sports and 700,000 streaming events. Handling over 6 billion HTTP requests daily and processing more than 2 million bets per hour at peak. Empowering employees to push boundaries and explore new ideas. Cultivating a culture that celebrates and rewards creativity. Breaking new ground in software innovation.
Company Description We’re one of the world’s leading online gambling companies, revolutionising the industry since 2000. Founded by Denise Coates CBE, we now employ over 10,000 people and serve over 120 million customers in 26 languages. We empower our employees to push boundaries and explore new ideas, cultivating a culture that celebrates and rewards creativity. This offers employees a wealth of growth opportunities, giving them the opportunity to make a real impact in the world of online gambling. As a forward-thinking company, we’re breaking new ground in software innovation too, redefining what’s possible for our global worldwide. Our focus on In-Play betting has solidified our market-leading position, featuring more than 1.38 million In-Play sporting events a year. With over 750 concurrent sporting fixtures at peak and more live sports streamed than anyone else in Europe (750,000), we handle over 6 million HTTP requests daily and process more than 1.5 million bets per hour at peak. Job Description As a Data Engineer, you will play a key role in creating, migrating, and maintaining regulatory reporting systems on Google Cloud Platform (GCP). The Regulatory Data team is a crucial function of the Data Engineering department, responsible for building and maintaining near real-time and batch reporting systems to fulfil regulatory requirements. The team utilises complex Business logic to transform internal data models into those required by industry regulators and third parties. Using your passion for delivering innovative solutions you will support our continued migration from on-premise SQL Server systems to GCP as the team creates new GCP reporting systems You will support this migration while also developing new cloud-native solutions and exploring AI-driven development and automation to improve efficiency, consistency and reliability across the team's processes. Qualifications - Experience with Google BigQuery. - SQL development experience, with the ability to write complex queries, stored procedures, and performance-tune data-intensive workloads. - Familiarity with GCP services such as Cloud Storage, Cloud Functions, Pub/Sub, or Cloud Composer. - Experience with data pipeline development (ETL/ELT) and orchestration tooling. - Exposure to Infrastructure as Code such as Terraform, and CI/CD pipelines such as Git, GitLab. - Methodical, with high attention to detail and the ability to break down complex requirements into simple solutions. - Interest in AI-assisted development tooling, such as Claude Code. - Ability to work to deadlines in a fast-paced, reactive environment. - SQL Server experience to support legacy systems during migration. Additional Information - Developing and maintaining regulatory data submission systems, both existing on-premise and new cloud-based solutions in GCP/BigQuery. - Building and maintaining automated data pipelines for ingestion, transformation, and loading of regulatory data. - Supporting the migration of legacy SQL Server solutions to BigQuery, working closely with seniors and technical leads on architecture and data modelling. - Implementing data validation, monitoring, and alerting to ensure accuracy, consistency, and reliability of regulatory submissions. - Leveraging AI tooling, such as Claude Code, and contributing to the team's automation strategies to reduce manual effort and improve quality. - Participating in code reviews and adhere to departmental standards and the development process. - Conducting QA to ensure solutions are accurate, efficient, and performant. - Creating and maintaining relevant documentation. - Collaborating with other team members to deliver high-quality solutions within required timeframes. By applying to us you are agreeing to share your Personal Data in accordance with our Recruitment Privacy Notice - https://www.bet365careers.com/privacy-policy At bet365, we're committed to creating an environment where everyone feels welcome, respected and valued. Where all individuals can grow and develop, regardless of their background. We're Never Ordinary, and we're always striving to be better. If you need any adjustments or accommodations to the recruitment process, at either application or interview, please don’t hesitate to reach out.
• Pipeline Development and Optimization: Build and maintain reliable, scalable ETL/ELT pipelines using modern tools and best practices, ensuring efficient data flow for analytics and insights. • Data Modeling and Transformation: Design and implement effective data models that support business needs, enabling high-quality reporting and downstream analytics. • Collaboration Across Teams: Work closely with data analysts, product managers, and other engineers to understand data requirements and deliver solutions that meet the needs of the business. • Ensuring Data Quality: Develop and apply data quality checks, validation frameworks, and monitoring to ensure the consistency, accuracy, and reliability of data. • Performance and Efficiency: Identify and address performance issues in pipelines, queries, and data storage. Suggest and implement optimizations that enhance speed and reliability. • Security and Compliance: Follow data security best practices and ensure pipelines are built to meet data privacy and compliance standards. • Innovation and Continuous Improvement: Test new tools and approaches by building Proof of Concepts (PoCs) and conducting performance benchmarks to find the best solutions. • Automation and CI/CD Practices: Contribute to the development of robust CI/CD pipelines (GitLab CI or similar) for data workflows, supporting automated testing and deployment.
• Pipeline Development and Optimization: Build and maintain reliable, scalable ETL/ELT pipelines using modern tools and best practices, ensuring efficient data flow for analytics and insights. • Data Modeling and Transformation: Design and implement effective data models that support business needs, enabling high-quality reporting and downstream analytics. • Collaboration Across Teams: Work closely with data analysts, product managers, and other engineers to understand data requirements and deliver solutions that meet the needs of the business. • Ensuring Data Quality: Develop and apply data quality checks, validation frameworks, and monitoring to ensure the consistency, accuracy, and reliability of data. • Performance and Efficiency: Identify and address performance issues in pipelines, queries, and data storage. Suggest and implement optimizations that enhance speed and reliability. • Security and Compliance: Follow data security best practices and ensure pipelines are built to meet data privacy and compliance standards. • Innovation and Continuous Improvement: Test new tools and approaches by building Proof of Concepts (PoCs) and conducting performance benchmarks to find the best solutions. • Automation and CI/CD Practices: Contribute to the development of robust CI/CD pipelines (GitLab CI or similar) for data workflows, supporting automated testing and deployment.

