Job Closed
This listing is no longer active.
Bringing down borders in technology.
Senior Data Engineer – Microsoft Fabric
Location
United States
Posted
90 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer – Microsoft Fabric
Upstart 13
• Own the design, build-out, and evolution of our data platform on Microsoft Fabric • Design and implement end-to-end ELT pipelines using Data Factory, Spark notebooks (PySpark/Spark SQL), and Dataflows Gen2 • Build and maintain a medallion architecture (Bronze → Silver → Gold) • Implement data validation, schema enforcement, and automated quality checks; partner with governance on lineage • Manage Fabric platform operations, capacity monitoring, Git-based CI/CD for notebooks and pipelines • Deliver well-structured Gold-layer tables optimized for Direct Lake • Design and implement RTM Isolation strategies using CDC • Collaborate with BI stakeholders to deliver AI-Ready Gold-layer tables
Job Requirements
- 6+ years building production ETL/ELT pipelines in cloud data platforms
- Strong hands-on experience with Microsoft Fabric (Lakehouses, Warehouses, Data Factory, Spark notebooks, OneLake)
- Deep understanding of lakehouse and medallion architecture patterns
- Proficient in PySpark, Spark SQL, and/or T-SQL for data transformation and optimization
- Experience with Delta Lake table management (partitioning, optimization, incremental loads, merge/upsert patterns)
- Familiarity with dimensional modeling and star-schema design principles
- Working knowledge of Git-based CI/CD for data artifacts (notebooks, pipelines, TMDL/PBIP a plus)
- Comfortable working with the SQL analytics endpoint and understanding Direct Lake requirements
Benefits
- 20+ Paid time off days
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Lead Data Engineer, Databricks – Manufacturing/MES
CompuGainDeliver game-changing modern applications with Unisys
• Design, build, and optimize scalable data pipelines using Azure Databricks (PySpark, Spark SQL, Delta Lake)• Integrate and process MES data, shop-floor data, IoT/sensor data, and ERP systems• Architect modern data solutions supporting manufacturing analytics and operational reporting• Lead data engineering best practices including performance tuning, optimization, and governance• Work closely with manufacturing stakeholders to translate MES data requirements into scalable data models• Ensure high data quality, reliability, and production-grade deployments.
Pay: $110,000 - $125,000 per year Job Description: Python Developer The Python Developer is responsible for designing, developing, and implementing Python based automation to execute the reconciliation process. They are responsible for building robust, scalable ETL data pipelines that extract, transform, and load data from multiple sources into an integrated platform. This role develops, tests, and deploys Python automation scripts and workflows, ensuring accuracy, efficiency, and reliability throughout the reconciliation lifecycle. Working within cloud and enterprise data environments, they collaborate closely with technical leads and stakeholders to translate requirements into well-architected, maintainable solutions and continuously enhance automation performance. Key Responsibilities: - Design, develop, and maintain robust ETL pipelines for data integration across multiple sources. - Refactor existing code and automated processes into Python, validating functionality through detailed test cases and user feedback. - Implement scalable data solutions using Python and cloud-based platforms such as Databricks and Apache Airflow. - Collaborate with data scientists and analysts to ensure seamless data accessibility and usability. - Optimize data architecture and workflows to enhance efficiency, performance, and security. - Troubleshoot data pipeline issues and ensure data integrity across systems. The ideal candidate must have the following experience, background, and credentials: - 3+ years of experience in data and automation engineering with a strong focus on Python - 3+ years of experience developing, maintaining, and administrating SQL databases - 2+ years of experience working with cloud-based data platforms such as Databricks, Apache Airflow, or other AWS services (e.g. Lambda, S3) - Experience designing, testing, and deploying robust ETL workflows and integrate with enterprise data systems. - Be a U.S. citizen able to pass a background investigation by the client agency. DHS clearance with CBP/ICE, or DoD Top-Secret preferred. Preferred Skills: - Experience with Optical Character Recognition (OCR) and Document Understanding frameworks (e.g., AWS Textract, Tesseract, or layout-aware models) - Familiarity with DevOps practices such as version control, CI/CD, and containerized deployment. Job Type: Full-time Work Location: Remote Experience: - Python: 3 years (Required) Benefits: - 401(k) - 401(k) matching - Dental insurance - Employee assistance program - Flexible schedule - Flexible spending account - Health insurance - Life insurance - Professional development assistance - Tuition reimbursement - Vision insurance Application Question(s): - This position requires U.S. citizenship. Are you a U.S. citizen ? - Do you have a DHS or a DoD clearance ?
Pay: $120,000.00-$130,000.00 per year Job Description: PowerApps/Data Visualization Developer Constellation seeks a PowerApps/Data Visualization Developer to join our Homeland Security Team in Washington, DC. This role requires strong PowerApps development expertise alongside advanced dashboarding and data analysis skills. The analyst will design, build, and maintain PowerApps solutions and Power BI dashboards that integrate operational and financial data to create meaningful, actionable insights for stakeholders. This position is client-facing and requires excellent communication to translate complex technical outputs into clear business value. Key Responsibilities: - Design, develop, and maintain PowerApps applications to automate workflows and enhance operational processes. - Build, maintain, and optimize dashboards, reports, and data models using Power BI, Tableau, Qlik, or other visualization tools. - Collaborate with end users and technical resources to understand and integrate data outputs into visualizations effectively. - Translate complex data findings into clear, digestible insights for stakeholders to support data-driven decision-making. - Ensure data accuracy, consistency, and integrity across reports and dashboards. - Identify data attributes (e.g. financial, biometric, personnel-based) critical for performance analysis and trend identification. - Conduct training sessions as necessary for CBP and contractor personnel on data analysis tools and methodologies. The ideal candidate must have the following experience, background, and credentials: - At least 4 years of experience working with large datasets using Power BI, Tableau, Qlik, or other comparable dashboarding tools. - Proven experience developing PowerApps solutions for business process automation and data integration. - Strong proficiency in PowerApps and Power BI (or comparable dashboard tool) is required. Strong proficiency with Excel is also required. Experience with Power Automate and SharePoint is preferred. - Excellent communication skills with the ability to gather requirements, present solutions, and translate complex data findings into clear insights for non-technical stakeholders. - Bachelor’s degree in Data Science, Computer Science, Statistics, or a related field; - Certification in Power Platform or data analytics tools is a plus. - Be a U.S. citizen able to pass a background investigation by the client agency. DHS clearance with CBP/ICE or DoD Top-Secret preferred. Job Type: Full-time Work Location: Remote Education: - Bachelor's (Required) Experience: - Power BI OR Tableau OR Qlik: 4 years (Required) - PowerApps: 2 years (Required) Benefits: - 401(k) - 401(k) matching - Dental insurance - Flexible spending account - Health insurance - Life insurance - Paid time off - Vision insurance Application Question(s): - This position requires U.S. citizenship. Are you a U.S. citizen ? - Do you have a DHS or a DoD clearance ?
• As the Data Engineering Lead for B2B Data Products, you will help design, build, and scale data solutions that extend the capabilities of our B2B Identity platform. • Create high‑quality B2B data attributes, strengthen professional‑to‑consumer linkage processes, and champion engineering excellence across the product ecosystem. • Collaborate closely with partners across product, identity, and engineering to align goals, shape strategy, and deliver consistent, reliable data experiences. • Partner with product leaders to translate product strategy into technical workstreams and delivery plans. • Design and maintain data pipelines that process firmographic, professional, and consumer‑level data attributes. • Lead and mentor a distributed engineering team, including offshore team members, with clarity, structure, and support.



