At NetWitness, we believe in challenging the established mindsets, approaches, and product categories in the information security industry. Every product that we deliver to market is based on a core set of principles grounded in the major paradigm shifts in play and the implications that they have for our customers. Do the right thing – by our customers, employees, and shareholders...think long-term, but act with a sense of urgency. What we do matters – our work makes a difference in the world. We give a damn – about our customers, about what we’re doing, about each other...we’re in this together. We are a fun company – building cool products with technical insight that help our customers solve meaningful problems. Our mission is delighting our customers with everything we do. We provide thousands of customers around the world with essential security capabilities, leading with our Intelligence Driven Security Strategy and Vision, to protect their most valuable assets from cyber threats. With NetWitness’s award-winning products, organizations effectively detect, investigate, and respond to advanced attacks; reduce IP theft and cybercrime.
Data Engineer
Location
Egypt
Posted
4 days ago
Salary
0
Seniority
Mid Level
Job Description
Data Engineer
Partner One Capital
Role Description The Data Engineer operates within the framework established by the Lead — designing, building, and maintaining robust data pipelines and transformation logic that power analytics, compliance, and operational reporting across the Mortgage Cadence Platform. The role is execution-focused with increasing ownership of end-to-end data workflows as familiarity with the platform grows. Strong SQL, ETL, and data quality skills are required; the ability to build reports and leverage semantic models is secondary to data engineering excellence. - Design and build extraction, transformation, and loading (ETL) pipelines using Microsoft Fabric (Dataflow Gen2, Notebooks, or equivalent tools) - Write optimized SQL queries and transformations for data ingestion from designated source systems - Apply data quality rules and validation logic at each pipeline stage - Implement incremental loads and manage refresh schedules for performance - Escalate to Lead for architectural decisions or complex transformation patterns - Define and implement data quality checks at ingestion, transformation, and output stages - Perform ongoing data validation to ensure pipeline outputs align with business logic and source system expectations - Identify, document, and escalate data quality issues with root cause analysis - Maintain data quality dashboards and SLA monitoring - Support UAT for new data sources or transformation logic - Build and maintain data transformations using Power Query, SQL, or Python as appropriate - Develop dimensional models and define aggregation logic aligned with analytics requirements - Optimize data structures for performance and maintainability - Document transformation logic, lineage, and assumptions per team standards - Troubleshoot pipeline failures and performance issues; coordinate resolution with IT/Engineering - Respond to data discrepancy reports from business users and analysts - Maintain documentation of data sources, data dictionaries, and transformation specifications - Support capacity planning and optimization of Fabric environments and pipelines - Collaborate with Lead to define semantic models and calculated metrics Qualifications - Advanced SQL — query optimization, window functions, performance tuning, debugging complex transformations - Proficient with Microsoft Fabric — (Dataflow Gen2, Notebooks, Lakehouse) OR equivalent ETL tools (Python, dbt, Talend, Informatica) - Strong understanding of relational database design and dimensional modeling - Power Query / M — complex data shaping, merging, error handling, and transformation logic - Python or similar scripting language — data manipulation, pipeline automation - Git/version control basics — able to collaborate on code and track changes - Data quality and testing frameworks — unit tests, assertions, validation rules - Ability to interpret business requirements and design efficient data solutions - Data governance mindset — understands data lineage, documentation, and quality standards - Proactive about identifying edge cases and potential data issues - Mortgage/lending domain familiarity preferred; willingness to learn domain required - Works effectively within defined standards and escalates architectural questions to Lead - Able to balance speed with quality; advocates for technical excellence
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Leads projects for design, development and maintenance of a data and analytics platform. • Effectively and efficiently process, store and make data available to analysts and other consumers. • Works with key business stakeholders, IT experts and subject-matter experts to plan, design and deliver optimal analytics and data science solutions. • Works on one or many product teams at a time. • Designs and automates deployment of our distributed system for ingesting and transforming data from various types of sources (relational, event-based, unstructured). • Designs and implements framework to continuously monitor and troubleshoot data quality and data integrity issues. • Implements data governance processes and methods for managing metadata, access, retention to data for internal and external users. • Designs and provides guidance on building reliable, efficient, scalable and quality data pipelines with monitoring and alert mechanisms that combine a variety of sources using ETL/ELT tools or scripting languages. • Designs and implements physical data models to define the database structure. • Optimizing database performance through efficient indexing and table relationships. • Participates in optimizing, testing, and troubleshooting of data pipelines. • Designs, develops and operates large scale data storage and processing solutions using different distributed and cloud based platforms for storing data (e.g. Data Lakes, Hadoop, Hbase, Cassandra, MongoDB, Accumulo, DynamoDB, others). • Uses innovative and modern tools, techniques and architectures to partially or completely automate the most-common, repeatable and tedious data preparation and integration tasks in order to minimize manual and error-prone processes and improve productivity. • Assists with renovating the data management infrastructure to drive automation in data integration and management. • Ensures the timeliness and success of critical analytics initiatives by using agile development technologies such as DevOps, Scrum, Kanban Coaches and develops less experienced team members.
• Design, build, and maintain scalable ETL/ELT pipelines. • Integrate data from internal and external systems (CRMs, LMS, marketing platforms, databases, etc.). • Develop and maintain API integrations (REST/JSON, OAuth, etc.). • Automate recurring data processes using Python (or similar scripting languages). • Design and maintain data models optimized for analytics and operational reporting. • Work with cloud data warehouses (e.g., BigQuery, Snowflake) to ensure efficient storage and retrieval. • Implement data validation and monitoring mechanisms to ensure accuracy and reliability. • Proactively monitor and resolve data pipeline failures. • Create dashboards and data applications using low-code/no-code platforms (e.g., Power BI, Retool, Coda, Tableau, Google Sheets). • Translate complex datasets into clear, actionable insights for business stakeholders. • Collaborate with analysts, data scientists, and business teams to define KPIs and reporting requirements. • Suggest improvements to existing data flows and business processes. • Maintain structured and comprehensive technical documentation (pipelines, integrations, data definitions, architecture). • Participate in cross-functional discussions to align technical implementation with business needs.
• Take ownership of end-to-end data engineering projects. • Design, build, maintain, and optimise scalable data pipelines using Dagster, Python, and Snowflake. • Write clean, efficient, well-documented code and ensuring quality and scalability of the codebase • Ensure high standards of data quality are maintained through extensive unit testing, integration testing and data monitoring. • Design and develop complex data models aligned with industry best practices. • Participate in code reviews, providing constructive feedback to maintain high standards. • Assist with the design and implementation of our data architecture. • Stay up-to-date with industry trends and emerging technologies.
Role Description We’re looking for a Data Engineer to drive the next phase of our data strategy at Midnite. You’ll work across the full data lifecycle from ingestion and modelling to orchestration, monitoring, and analytics enablement, while also helping shape engineering best practices. Our tech stack includes: - Python - Docker - Dagster - dbt - Fivetran - Apache Iceberg - Snowflake - S3 - ECS You will: - Take ownership of end-to-end data engineering projects. - Design, build, maintain, and optimise scalable data pipelines using Dagster, Python, and Snowflake. - Write clean, efficient, well-documented code and ensure quality and scalability of the codebase. - Ensure high standards of data quality are maintained through extensive unit testing, integration testing, and data monitoring. - Design and develop complex data models aligned with industry best practices. - Participate in code reviews, providing constructive feedback to maintain high standards. - Assist with the design and implementation of our data architecture. - Stay up-to-date with industry trends and emerging technologies. Qualifications - 3+ years of experience in a similar role, with a track record of autonomous work and impact. - Strong expertise in Python and SQL (Snowflake preferred). - Experience using monitoring and alerting tools. - Deep understanding of engineering best practices (e.g. unit testing, writing DRY code). - Hands-on experience with a modern data orchestrator (e.g. Airflow, Dagster, Prefect). - Knowledge of data modelling concepts and methodologies (Kimball dimensional modelling, data vault, etc.). - Proficiency with dbt. - Comfortable working with Git and version control workflows. Requirements - Experience with Snowflake. - Experience with AWS’s data ecosystem. - Experience implementing low-latency analytics at scale. - Experience with infrastructure-as-code (Terraform preferred). - Experience delivering full end-to-end data products, including data visualisation and BI. Benefits - Private health insurance with zero excess, including optical cover and optional dental. - Income protection to protect your earnings and give you peace of mind. - Tenure holiday policy: After three years, you receive an extra two days leave, increasing to 30 days annually after five years. - Flexible working and a fully supported home office setup. - Nursery salary sacrifice scheme helping parents save thousands each year on nursery fees. - Salary sacrifice schemes for tech and holidays. - Retail discounts and subscription perks across a wide range of brands. - Quarterly team socials to connect, celebrate, and have fun together.
