LanceSoft, Inc. logo
LanceSoft, Inc.

Global Workforce Solutions With A Human Touch. #Spearheadingfuture

Data Engineer

Data EngineerData EngineerFull TimeRemoteMid LevelTeam 5,001-10,000Since 2000Company SiteLinkedIn

Location

United States

Posted

14 days ago

Salary

$76 - $83 / hour

Seniority

Mid Level

Job Description

Data Engineer

LanceSoft, Inc.

Role Description Data Analyst - Finance Data Platform - Purpose: Support data validation, data reconciliation, and reports development for the Finance mainframe application modernization and data platform migration to modern GCP-based data platform. - Outcome: Identify data variance between legacy and modern platforms for modern Finance applications and 2 quarter parallel operation. Collaborate with product and business teams to identify root causes for the variances and resolve/document the variance to assist on Go-Live decision. - Core Skill: Hands-on SQL experience with big data platforms/databases like Google BigQuery, Hadoop or similar data platform. Qualifications - Senior level (5+ years) hands-on SQL experience with big data platforms/databases like Google BigQuery, Hadoop or similar data platform. - Senior experience with ETL and data pipeline tools such as Dataproc, Airflow, Dataform, dbt, or similar tools. - Good experience with Python/Pyspark coding for data movement and analysis. - Experience with data visualization and reporting tools such as Power BI, Looker, and LookML for dashboard development, KPI reconciliation, semantic layer validation, and data analysis. - Strong experience in data analysis, data validation, source-to-target reconciliation, duplicate detection, reject/error validation, audit/control total validation, and business rule validation. - Ability to analyze mapping documents, reporting requirements, transformation logic, data lineage, and data quality rules. - Experience creating reconciliation reports, validation summaries, exception reports, defect logs, and business sign-off documentation. - Strong communication and collaboration skills with the ability to work closely with engineering, product, finance, business, and legacy system teams. Requirements - Develop queries and dataset required to validate legacy vs modern applications datasets to identify data variances. - Develop reports using end user visualization tools to present the variances and trends between sources. - Document data variances and identified root causes in collaboration with Product Manager/Engineers/Business Teams and get business sign off for known variances. - Build dashboards for data variance trend for the stakeholders and assist on determining tolerance thresholds for Go Live decisions. - 5+ years working in data analytics role. - 1-3 years of experience working in agile team. Benefits - Four options of medical Insurance - Dental and Vision Insurance - 401k Contributions - Critical Illness Insurance - Voluntary Permanent Life Insurance - Accident Insurance - Other Employee Perks

Related Categories

Related Job Pages

More Data Engineer Jobs

Data Scientist Architect

Navtech, Inc.

NAVTECH INC 1600 Golf Road. Suite 1200, Rolling Meadows, IL 60008 Ph: (224) 348-1340 Email: alex@navtechusa.com Website: www.navtechusa.com E-Verified Company

Data Engineer15 days ago

Role Description Designs and develops scalable solutions using AI tools and machine-learning models. - Performs research and testing to develop machine learning algorithms and predictive models. - Utilizes big data computation and storage tools to create prototypes and datasets. - Conducts model training and evaluation. - Integrates, tests, tunes, and monitors solutions. - Proficient with multiple AI tools such as Python, Java, or R and machine learning frameworks like Spark, TensorFlow, or scikit-learn. Requires a master's degree in computer science, mathematics, engineering or equivalent. Typically reports to a manager or head of a unit/department. - P05-Expert: Works autonomously. Goals are generally communicated in "solution" or project goal terms. - May provide a leadership role for the work group through knowledge in the area of specialization. - Works on advanced, complex technical projects or business issues requiring state of the art technical or industry knowledge. - Typically requires 10+ years of related experience. Qualifications - Master's degree in computer science, mathematics, engineering or equivalent. - 10+ years of related experience. Requirements - Proficiency in AI tools such as Python, Java, or R. - Experience with machine learning frameworks like Spark, TensorFlow, or scikit-learn. - Ability to work autonomously and lead a work group. - Expertise in advanced technical projects or business issues.

United States
Job Closed
Paires logo

Data Engineer

Paires

Fundraising AI for breakout founders and the investors behind them

Data Engineer15 days ago
Full TimeRemoteTeam 51-200

• Own the database consisting of Postgres and Supabase, managing schema design, modeling, and scaling performance. • Ensure data quality end-to-end via validation gates, deduplication, and entity resolution. • Oversee the communications layer, linking raw emails and call transcripts to the right people and companies. • Develop ingestion and enrichment pipelines for funding rounds and company research at scale. • Manage the knowledge graph comprising companies, investors, funding rounds, and news as entities and relationships. • Create a unified data layer that serves every campaign and product feature.

Canada
CA$150K - CA$250K / year
Full TimeRemoteTeam 51-200

Role Description Forge requires a Mid Data Engineer to support legacy-to-modern data transformation in a secure AWS environment for a DoW customer. The role will develop batch and event-driven pipelines, automate data quality and testing, integrate with application services, and provide observable, recoverable, high-quality data flows across mission and external interfaces. Key Responsibilities - Build secure Python and AWS ETL/ELT pipelines for ingestion, transformation, reconciliation, and delivery. - Develop and evolve relational data models, schemas, indexes, constraints, views, and access patterns for MariaDB, PostgreSQL, or comparable platforms. - Develop data workflows and interfaces using Python on AWS Lambda and PySpark for event-driven, batch, and distributed transformation workloads. - Create automated data-quality checks for accuracy, completeness, consistency, timeliness, uniqueness, and business-rule conformance. - Implement source-to-target mapping, lineage, auditability, restartability, exception handling, and controlled replay. - Develop parity tests that compare legacy and modern processing outcomes and document the disposition of intentional differences. - Tune SQL and pipeline performance for high-volume batch and near-real-time workloads while protecting transactional integrity. - Implement monitoring, logging, alerting, and operational dashboards for pipeline health, latency, failures, and data quality. - Automate CI/CD, version-controlled data changes, deployments, rollback, and operational recovery controls. - Collaborate with architects, mission SMEs, Appian developers, testers, security personnel, and interface partners. Qualifications - Ability to think strategically, act tactically, and demonstrate strong analytical and critical-thinking skills. - Build strong cross-group working relationships and demonstrate exceptional organizational skills and attention to detail. - Thrive and succeed in an entrepreneurial environment and not be hindered by ambiguity or competing priorities. - Self-managing candidates who enjoy working collaboratively in a fast-paced environment and with dynamic teams. Requirements - U.S. Citizen (Authorization to Work in the U.S. will not suffice); previous professional experience supporting the U.S. Federal Government, either as a federal employee or contractor, is required. - 4+ years of professional experience in data engineering, database development, or data-platform delivery. - Bachelor's degree in Computer Science, Information Systems, Data Engineering, or equivalent, OR 4 additional years of relevant professional experience in lieu of a degree. - Advanced Python software-engineering skills and experience building AWS Lambda functions and PySpark data-transformation pipelines. - Experience building, testing, and operating production ETL/ELT pipelines with automated data-quality controls. - Experience with data modeling, schema migration, source-to-target mapping, lineage, reconciliation, and data-quality automation. - Experience integrating data platforms with REST APIs, application services, file exchanges, and event-driven interfaces. - Experience with Git, CI/CD, automated testing, logging, monitoring, performance tuning, and production support. - Active CompTIA Security+ or equivalent DoW-approved baseline cybersecurity certification, or ability to obtain within the first 30 days of starting. - Active Tier 2 background investigation or higher, completed or favorably adjudicated within the previous 18 months. Highly Desired Qualifications - Experience using Palantir Foundry for data integration, transformation, lineage, governance, and operational workflows. - Active Secret security clearance preferred. - Previous professional experience supporting a DoW organization, mission, or customer is strongly preferred; experience in modernizing COBOL flat files or legacy relational data into a modern relational architecture. - Experience integrating with Appian, Python microservices, financial transactions, logistics workflows, or high-volume external interfaces. - Experience serving as a technical team lead, mentoring junior engineers, or assisting teammates across delivery tasks. - Experience with BI, analytics, archival, records-retention, or NARA-aligned data lifecycle requirements. Benefits - Complete Flextime - 401k With Employer Matching - Healthcare, Including Medical, Dental, and Vision - Health Savings Account (HSA) And Pre-Tax Premium Options - Supplementary healthcare and family support - Extended Short-Term Disability and Long-Term Disability - Healthcare Insurance Deductible Paydown - Health and Wellness Programs - Tuition Reimbursement, Student Loan Repayment, and Education & Training Stipends - Cell Phone / Internet Stipends - College Saving Plans with Employer Contributions - Alternative Work Locations and Tele-Commuting - Employee Referral Awards - Retention, Signing & Performance Bonuses - Commuter Benefits - Paid Sabbatical

United States
$130K - $155K / year
Weekday (YC W21) logo

Senior Data Engineer, Microsoft Fabric Engineer

Weekday (YC W21)

We are a Y-Combinator-backed startup building your AI-powered Recruiter Agent

Data Engineer15 days ago
Full TimeRemoteTeam 11-50Since 2021H1B No Sponsor

• Design, develop, and maintain scalable data pipelines using Microsoft Azure Fabric, Databricks, and Azure data services. • Build and optimize robust ETL/ELT processes for structured, semi-structured, and unstructured data across enterprise environments. • Collaborate with business stakeholders to understand data requirements and translate them into scalable technical solutions. • Develop and maintain reliable data integration workflows that ensure data accuracy, consistency, and availability. • Optimize data processing pipelines for performance, scalability, cost efficiency, and operational reliability. • Leverage AI-assisted development tools and modern engineering practices to accelerate solution delivery and improve code quality. • Monitor, troubleshoot, and resolve production data pipeline issues while proactively identifying opportunities for automation and optimization. • Work closely with architects, analysts, developers, and cross-functional teams throughout the project lifecycle to deliver high-quality data solutions. • Implement best practices for data engineering, governance, documentation, testing, and operational support. • Take ownership of project deliverables by ensuring quality, meeting timelines, communicating risks proactively, and continuously improving engineering processes.

India
Rs1,100K - Rs5,000K / year