Job Closed
This listing is no longer active.
Deliver game-changing modern applications with Unisys
Lead Data Engineer, Databricks – Manufacturing/MES
Location
California
Posted
92 days ago
Salary
0
Seniority
Senior
Job Description
Lead Data Engineer, Databricks – Manufacturing/MES
CompuGain
• Design, build, and optimize scalable data pipelines using Azure Databricks (PySpark, Spark SQL, Delta Lake)• Integrate and process MES data, shop-floor data, IoT/sensor data, and ERP systems• Architect modern data solutions supporting manufacturing analytics and operational reporting• Lead data engineering best practices including performance tuning, optimization, and governance• Work closely with manufacturing stakeholders to translate MES data requirements into scalable data models• Ensure high data quality, reliability, and production-grade deployments.
Job Requirements
- Strong hands-on expertise in Azure Databricks (Delta Lake, PySpark, Spark architecture)
- Proven experience in Manufacturing domain, especially working with MES systems and production data
- Experience building real-time or near-real-time data pipelines
- Strong SQL and data modeling skills
- Experience with cloud platforms (Azure preferred)
- Ability to function as both technical architect and hands-on developer
- Excellent communication skills and ability to work in PST time zone
- Preferred: Experience with IoT data ingestion and streaming technologies
- Exposure to data warehousing and BI integration
- Experience leading small data engineering team
Benefits
- All your information will be kept confidential according to EEO guidelines.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Pay: $110,000 - $125,000 per year Job Description: Python Developer The Python Developer is responsible for designing, developing, and implementing Python based automation to execute the reconciliation process. They are responsible for building robust, scalable ETL data pipelines that extract, transform, and load data from multiple sources into an integrated platform. This role develops, tests, and deploys Python automation scripts and workflows, ensuring accuracy, efficiency, and reliability throughout the reconciliation lifecycle. Working within cloud and enterprise data environments, they collaborate closely with technical leads and stakeholders to translate requirements into well-architected, maintainable solutions and continuously enhance automation performance. Key Responsibilities: - Design, develop, and maintain robust ETL pipelines for data integration across multiple sources. - Refactor existing code and automated processes into Python, validating functionality through detailed test cases and user feedback. - Implement scalable data solutions using Python and cloud-based platforms such as Databricks and Apache Airflow. - Collaborate with data scientists and analysts to ensure seamless data accessibility and usability. - Optimize data architecture and workflows to enhance efficiency, performance, and security. - Troubleshoot data pipeline issues and ensure data integrity across systems. The ideal candidate must have the following experience, background, and credentials: - 3+ years of experience in data and automation engineering with a strong focus on Python - 3+ years of experience developing, maintaining, and administrating SQL databases - 2+ years of experience working with cloud-based data platforms such as Databricks, Apache Airflow, or other AWS services (e.g. Lambda, S3) - Experience designing, testing, and deploying robust ETL workflows and integrate with enterprise data systems. - Be a U.S. citizen able to pass a background investigation by the client agency. DHS clearance with CBP/ICE, or DoD Top-Secret preferred. Preferred Skills: - Experience with Optical Character Recognition (OCR) and Document Understanding frameworks (e.g., AWS Textract, Tesseract, or layout-aware models) - Familiarity with DevOps practices such as version control, CI/CD, and containerized deployment. Job Type: Full-time Work Location: Remote Experience: - Python: 3 years (Required) Benefits: - 401(k) - 401(k) matching - Dental insurance - Employee assistance program - Flexible schedule - Flexible spending account - Health insurance - Life insurance - Professional development assistance - Tuition reimbursement - Vision insurance Application Question(s): - This position requires U.S. citizenship. Are you a U.S. citizen ? - Do you have a DHS or a DoD clearance ?
Pay: $120,000.00-$130,000.00 per year Job Description: PowerApps/Data Visualization Developer Constellation seeks a PowerApps/Data Visualization Developer to join our Homeland Security Team in Washington, DC. This role requires strong PowerApps development expertise alongside advanced dashboarding and data analysis skills. The analyst will design, build, and maintain PowerApps solutions and Power BI dashboards that integrate operational and financial data to create meaningful, actionable insights for stakeholders. This position is client-facing and requires excellent communication to translate complex technical outputs into clear business value. Key Responsibilities: - Design, develop, and maintain PowerApps applications to automate workflows and enhance operational processes. - Build, maintain, and optimize dashboards, reports, and data models using Power BI, Tableau, Qlik, or other visualization tools. - Collaborate with end users and technical resources to understand and integrate data outputs into visualizations effectively. - Translate complex data findings into clear, digestible insights for stakeholders to support data-driven decision-making. - Ensure data accuracy, consistency, and integrity across reports and dashboards. - Identify data attributes (e.g. financial, biometric, personnel-based) critical for performance analysis and trend identification. - Conduct training sessions as necessary for CBP and contractor personnel on data analysis tools and methodologies. The ideal candidate must have the following experience, background, and credentials: - At least 4 years of experience working with large datasets using Power BI, Tableau, Qlik, or other comparable dashboarding tools. - Proven experience developing PowerApps solutions for business process automation and data integration. - Strong proficiency in PowerApps and Power BI (or comparable dashboard tool) is required. Strong proficiency with Excel is also required. Experience with Power Automate and SharePoint is preferred. - Excellent communication skills with the ability to gather requirements, present solutions, and translate complex data findings into clear insights for non-technical stakeholders. - Bachelor’s degree in Data Science, Computer Science, Statistics, or a related field; - Certification in Power Platform or data analytics tools is a plus. - Be a U.S. citizen able to pass a background investigation by the client agency. DHS clearance with CBP/ICE or DoD Top-Secret preferred. Job Type: Full-time Work Location: Remote Education: - Bachelor's (Required) Experience: - Power BI OR Tableau OR Qlik: 4 years (Required) - PowerApps: 2 years (Required) Benefits: - 401(k) - 401(k) matching - Dental insurance - Flexible spending account - Health insurance - Life insurance - Paid time off - Vision insurance Application Question(s): - This position requires U.S. citizenship. Are you a U.S. citizen ? - Do you have a DHS or a DoD clearance ?
• As the Data Engineering Lead for B2B Data Products, you will help design, build, and scale data solutions that extend the capabilities of our B2B Identity platform. • Create high‑quality B2B data attributes, strengthen professional‑to‑consumer linkage processes, and champion engineering excellence across the product ecosystem. • Collaborate closely with partners across product, identity, and engineering to align goals, shape strategy, and deliver consistent, reliable data experiences. • Partner with product leaders to translate product strategy into technical workstreams and delivery plans. • Design and maintain data pipelines that process firmographic, professional, and consumer‑level data attributes. • Lead and mentor a distributed engineering team, including offshore team members, with clarity, structure, and support.
• Design, develop, optimize and maintain scalable data pipelines and transformations using Databricks, Apache Spark and SQL. • Implement data ingestion, transformation, and orchestration workflows to support back and where applicable real-time processing. • Perform data quality assurance activities, including identifying and resolving any inconsistencies in data flow, data outside legitimate ranges, and illogical data responses by developing data quality reports and investigation and resolution of data anomalies or errors by using a combination of software packages including SAS, Excel, and other software as warranted. • Use technical expertise, initiative, creativity, critical thinking, and strong communication and interpersonal skills daily to solve data quality problems in support of technical development efforts. • Implement data quality controls to ensure accuracy, completeness, and reliability of datasets. • Document data pipelines, transforms, business rules and data dependencies using appropriate technical documentation methods (e.g., data flow diagrams, data dictionaries, etc.). • Serve as liaison and coordinate with a multi-disciplinary team. • Collaborate with the program team to identify opportunities for process improvements, making strategic adjustments, and exploit opportunities focused on maximizing programmatic impact. • Communicate data issues, risks, and remediation approaches clearly to technical and non-technical team members.



