Job Closed

This listing is no longer active.

Subject Matter Expert – Data Engineering

Data EngineerData EngineerFull TimeRemoteLeadTeam 1,001-5,000H1B No SponsorCompany SiteLinkedIn

Location

United States

Posted

21 days ago

Salary

0

Seniority

Lead

Bachelor Degree7 yrs expEnglishAzureJavaPythonScalaSparkSQL

Job Description

Subject Matter Expert – Data Engineering

Bart & Associates, Inc.

• Lead and provide expert support for data collection, data validation, data visualization, and analytics initiatives • Apply disciplined methodologies for the planning, analysis, design, and development of information systems on an enterprise-wide basis or across a business sector • Develop analytical techniques and methodologies to solve complex business and technical problems • Perform strategic systems planning, business information planning, and business analysis • Organize and analyze large volumes of structured and unstructured data sets using data analytical tools • Locate, access, merge, clean, and standardize data from multiple sources, and develop derived metrics • Create and implement data collection and analysis tools using programming languages such as Python, Databricks, SQL, Scala, R, and Java • Design, script, debug, and analyze data engineering solutions • Implement and create machine learning-based tools and processes • Apply distributed and parallel processing technologies (e.g., Spark) to handle big data analytics tasks involving large data volumes • Perform SQL Server data imports from CSV and TXT files • Leverage Excel and Google Suite for data analysis, reporting, and collaboration • Utilize supporting tools and platforms such as Pentaho (data import/transformation), Azure Data Studio, GitHub, and Smartsheet as needed • Document task requirements, work completed, processes, and technical details thoroughly • Communicate effectively with stakeholders across technical and business teams • Operate independently as a subject matter expert in a fast-paced, entrepreneurial environment

Job Requirements

  • Bachelor’s or master’s degree in computer science, Computer Science, Data Science, Engineering, or a related field.
  • Minimum of seven (7) years experience, of which at least three (3) years must be specialized.
  • Knowledge in organizing and analyzing large amounts of structured and unstructured data sets using data analytical tools, to include experience locating, accessing, merging, cleaning, and standardizing data, and developing derived metrics.
  • Strong proficiency in creating and implementing data collection and analysis tools while utilizing programming languages and environments, including Python, Databricks, SQL, Scala, R, and Java.
  • Knowledge of design, scripting, debugging, and analysis.
  • Experience in the implementation and creation of machine learning-based tools or processes.
  • Knowledge in distributed & parallel processing (e.g., Spark) and the computational power required to handle most big data analytics tasks that can process large volumes of data.

Benefits

  • B&A is proud to offer three robust individual and family medical plans to full time employees, including a Health Savings Account (HSA) option as well as two tiers of dental coverage, vision, life & AD&D, disability, accident, hospital indemnity, and critical illness insurance.
  • In addition to these benefits, B&A employees enjoy paid time off, B&A sponsored trainings and certifications, pet insurance benefits, commuter transit benefits and a free subscription to a virtual exercise platform (NEOU).
  • A formal mentorship program
  • Job shadowing and cross training opportunities
  • Employee Assistance Program (EAP) - Access to various support resources to include counseling, legal guidance, financial planning, and more.
  • Monthly teambuilding events
  • B&A Annual Wellness Challenges: #StepWithB&A, #WalkDuringLunchWithB&A, #VolunteeringWithB&A, #ExerciseDuringLunchWithB&A, and more

Related Categories

Related Job Pages

More Data Engineer Jobs

EXL logo

Senior Data Engineer – Consent

EXL

We make sense of data to drive your business forward. #MakeSenseofData #DriveYourBusinessForward #PartnerYourWay

Data Engineer21 days ago
Full TimeRemoteTeam 10,001+H1B No Sponsor

• Design and build robust, scalable data transformation pipelines using SQL, DBT, and Jinja templating • Develop and maintain data architecture and standards for Data Integration and Data Warehousing projects using DBT and Amazon Redshift • Monitor and ensure the smooth operation of data pipelines between OneTrust (OT) application and external data platforms (ESPs, CRMs, etc.). • Collaborate with cross-functional teams to gather requirements and deliver dimensional data models that serve as a single source of truth • Assist in the ongoing data integration tasks (e.g., connectors, APIs) between OT and various business systems. • Own the full stack of data modeling in DBT to empower analysts, data scientists, and BI engineers • Enhance and maintain the analytics codebase, including DBT models, SQL scripts, and ERD documentation. • Familiarize regulatory compliance guidelines across different geographical areas (CCPA, GDPR) and integrate them into current and new pipelines for consent collection. • Ensure data quality, governance alignment, and operational readiness of data pipelines • Apply software engineering best practices such as version control, CI/CD, and code reviews • Optimize SQL queries for performance, scalability, and maintainability across large datasets • Implement best practices for SQL performance tuning, including partitioning, clustering, and materialized views • Build and manage infrastructure as code using AWS CDK for scalable and repeatable deployments. Integrate and automate deployment workflows using AWS CodeCommit, CodePipeline, and related DevOps tools • Support Agile development processes and collaborate with offshore teams

Canada
Job Closed
EXL logo

AWS Data Architect

EXL

We make sense of data to drive your business forward. #MakeSenseofData #DriveYourBusinessForward #PartnerYourWay

Data Engineer21 days ago
Full TimeRemoteTeam 10,001+H1B No Sponsor

• Develop and maintain a comprehensive data architecture and cloud strategy that aligns with the organization's goals and needs. • Design, implement, and manage cloud-based data infrastructure on AWS, ensuring scalability, reliability, and cost-efficiency. • Utilize AWS services (S3, Glue, EMR, Redshift, Lambda, Kinesis, MWAA, etc.) to build and optimize data pipelines and storage solutions. • Champion the use of data lakehouse architecture and optimize its performance for analytical and operational workloads. • Identify the gaps and opportunities in the current system and suggest/implement to optimise the processes and costs. • Lead and guide data engineering teams to develop, maintain, and optimize ETL processes for data ingestion, transformation, and loading. • Implement real-time data processing solutions using technologies such as Apache Kafka and AWS Kinesis. • Collaborate with data scientists, business stakeholders and analysts to ensure data availability and quality, enabling effective analytics and reporting. • Leverage DBT for data modelling and transformation to support self-service analytics and data governance. • Architect and implement data integration solutions for API ingestion, enabling data from diverse sources to be captured, transformed, and ingested into our data lakehouse. • Utilize Airbyte and custom APIs to ensure efficient, reliable, and secure data transfers. • Manage data integration pipelines to support real-time and batch data processing. • Design, configure, and maintain workflow orchestration using Apache Airflow to automate ETL processes and data pipeline executions. • Monitor and optimize job scheduling, error handling, and performance of data workflows. • Implement data security protocols, access controls, and encryption to safeguard sensitive data, especially PIIs. • Ensure compliance with data privacy regulations and industry standards. • Collaborate with cross-functional teams to understand data requirements and provide data solutions to meet their needs. • Maintain comprehensive documentation for data engineering and data architecture processes and solutions. • Guide the team in setting up cloud Infra and automate using tools like terraform, cloud formation, Jenkins etc • Guide the operations team in setting up automated monitoring & alerts mechanism

United States
EXL logo

Senior Data Engineer

EXL

We make sense of data to drive your business forward. #MakeSenseofData #DriveYourBusinessForward #PartnerYourWay

Data Engineer21 days ago
Full TimeRemoteTeam 10,001+H1B No Sponsor

• Design and build robust, scalable data transformation pipelines using SQL, DBT, and Jinja templating • Develop and maintain data architecture and standards for Data Integration and Data Warehousing projects using DBT and Amazon Redshift • Collaborate with cross-functional teams to gather requirements and deliver dimensional data models that serve as a single source of truth • Own the full stack of data modeling in DBT to empower analysts, data scientists, and BI engineers • Enhance and maintain the analytics codebase, including DBT models, SQL scripts, and ERD documentation • Ensure data quality, governance alignment, and operational readiness of data pipelines • Apply software engineering best practices such as version control, CI/CD, and code reviews • Optimize SQL queries for performance, scalability, and maintainability across large datasets • Implement best practices for SQL performance tuning, including partitioning, clustering, and materialized views • Build and manage infrastructure as code using AWS CDK for scalable and repeatable deployments. Integrate and automate deployment workflows using AWS CodeCommit, CodePipeline, and related DevOps tools • Support Agile development processes and collaborate with offshore teams

United States
Modash logo

Senior Product Data Engineer

Modash

Influencer tools for growth focused marketers! 💌

Data Engineer21 days ago
Full TimeRemoteTeam 11-50H1B No Sponsor

• Start your day with a short standup • Heads-down focus time to plan, build, iterate, and launch • Minimal meetings — maximum ownership • Creating an understanding of the creators location, age, and interests at scale • Creating systems to extract collaborations between creators and brands from raw social data • Shaping the future of AI-assisted search, exploring how LLMs and embeddings can enhance search and recommendations.

Estonia