Job Closed

This listing is no longer active.

Ad Hoc LLC logo
Ad Hoc LLC

Digital-first government for the common good.

Data Engineer II, PySpark/Databricks

Data EngineerData EngineerFull TimeRemoteLeadTeam 501-1,000Since 2014H1B No SponsorCompany SiteLinkedIn

Location

United States

Posted

113 days ago

Salary

$90K - $110K / year

Seniority

Lead

Bachelor Degree8 yrs expExperience acceptedEnglishApacheAWSETLPySparkPythonSpark

Job Description

Data Engineer II, PySpark/Databricks

Ad Hoc LLC

• Build and maintain PySpark data pipelines in the Databricks environment • Optimize Spark jobs performance and resource usage, identifying and addressing bottlenecks and inefficiencies in backend systems • Design, develop, and maintain high-quality backend software components and services, ensuring functionality, performance, and scalability • Research and build proof of concepts in the data space • Write clean, well-structured, and maintainable code, adhering to established coding standards and best practices • Perform thorough code reviews, providing constructive feedback to peers and identifying potential risks or areas for improvement • Debug and resolve defects, proactively identifying and addressing potential issues before they impact users • Create and maintain comprehensive technical documentation • Actively participate in Agile ceremonies, such as stand-ups, sprint planning, and retrospectives, ensuring effective communication and collaboration across the team • Assist in the estimation, prioritization, and planning of development tasks, ensuring projects are delivered on time and within budget • Continuously evaluate and recommend new dataframe related technologies, frameworks, and tools, helping to drive innovation and keep the team up-to-date with industry trends • Engage in ongoing professional development to stay current with industry best practices, and share knowledge and insights with the team as appropriate • Assist in the implementation and maintenance of security, compliance, and governance policies within the Databricks and AWS environment to ensure adherence to industry standards and regulatory requirements.

Job Requirements

  • Bachelor's degree and 8 years of experience
  • Strong experience with Python / Apache Spark
  • Solid understanding of data modeling, ETL process, and distributed computing
  • Bachelor's degree in Computer Science, Computer Engineering or related field
  • Strong understanding of software design patterns, data structures, and algorithms
  • Experience with Agile development methodologies
  • Ability to work independently as well as in a team
  • Strong problem-solving and analytical skills
  • Strong verbal and written communication skills
  • Related experience in analytic programming, data extraction, querying databases/data warehouses and data analysis.

Benefits

  • Company-subsidized health, dental, and vision insurance
  • Flexible PTO
  • 401K with employer match
  • Paid parental leave after one year of service
  • Employee Assistance Program

Related Categories

Related Job Pages

More Data Engineer Jobs

Data Science Associate

Havas

Havas, formerly known as Euro RSCG 4D Impact, is a marketing and advertising agency that aims to establish innovative and meaningful connections between brands

Data Engineer113 days ago

Data Science Associate Location: Peru Full time job requisition id JR0094496 Job Description: Agency : Havas Media Group Job Description : Data Science Associate - Marketing Analytics & Advanced Measurement (Global CoE) Location: Perú Fully Remote. CSA, the data consultancy within Havas Media Group, is seeking a Data Science Associate with a focus on Marketing to join our growing team. The Data Science Associate will have a passion for leveraging data-driven insights, implementing innovative marketing strategies, and optimizing customer experiences. They will play a pivotal role in providing state-of-the-art, privacy-first solutions for our clients, working with both event-level and aggregated data to create actionable insights and drive marketing performance. Key Responsibilities - Perform data wrangling tasks, including data cleaning, transformation, and integration from multiple sources to ensure high-quality and accurate datasets for analysis. - Continuously research and learn about new technologies and statistical methodologies to stay current in the rapidly evolving field of data science and marketing analytics. - Assist in applying advanced statistical techniques and machine learning algorithms to analyze event-level and aggregated data, identify patterns, and provide actionable insights to drive marketing performance. - Support the implementation and evaluation of marketing models, such as customer segmentation, lift analysis, attribution modeling, and marketing mix models. Requirements: - Bachelor's or Master's degree in Data Science, Economy, Statistics, Computer Science, or a related field. - Proficiency in statistical programming languages and packages such as Python, Pandas, and sci-kit-learn. Knowledge of SQL is desirable. - Knowledge of statistics (e.g., correlation and regression, statistical significance tests, t-tests, z-score) and Machine Learning (Clustering, ensembles, SVM). - Proficiency in MS Office tools such as Excel. - Advanced English language skills (mandatory). - Ability to work in a team, even remotely with global teams. What We Offer: Global Exposure: Work with international teams and global brands. Long-Term Contract: Stability and the opportunity to grow within a global network. Remote Work Model: Virtual work environment with a healthy work-life balance. Health & Wellness: EPS 100% health insurance and wellness initiatives. Culture & Community: Inclusive, collaborative, and purpose-driven workplace. Competitive Compensation: Full time contract monthly salary + food benefit card If you are a Data Scientist with a passion for marketing analytics and are eager to make a significant impact on our clients' success, we would love to hear from you! Apply now to join the CSA team within Havas Media Group and take your career to new heights. Contract Type : Permanent Here at Havas across the group we pride ourselves on being committed to offering equal opportunities to all potential employees and have zero tolerance for discrimination. We are an equal opportunity employer and welcome applicants irrespective of age, sex, race, ethnicity, disability and other factors that have no bearing on an individual's ability to perform their job.

Peru

Data Architect

CellPoint Digital

What makes CellPoint Digital a leader in the payment landscape isn’t just our technology - it’s our people and how we work together. We’ve built a global community where diverse talents and perspectives unite to create innovative solutions. When you join us, you become part of something bigger: a collaborative culture that crosses borders and disciplines, bringing out the best in every team member to deliver breakthrough results for our clients and partners. Together, we are transforming the payments industry - challenging, supporting, and inspiring one another in the process.

Data Engineer113 days ago

Role Description Join us as a Data Architect (Team Lead) on our mission to turn payments into possibilities! You will lead the design and operation of our payments data platform , while managing and developing a team of data engineers and analytics engineers. This role owns both the technical architecture and the definition of truth across the organization—ensuring that data is reliable, consistent, and aligned to how the business makes decisions . You are the final authority on data definitions, system design, and data quality standards . We are a high-velocity team. You are expected to leverage AI tools to accelerate development, analysis, and problem-solving . How You Will Make an Impact: - Define and evolve the data platform architecture - Design scalable, resilient systems for ingesting, processing, and serving payments data - Establish patterns and standards across pipelines, storage, and access layers - Create alignment on metrics across the organization - Standardize KPI definitions across product, engineering, operations, and leadership - Eliminate conflicting interpretations and ensure consistent decision-making - Act as the final arbiter of data definitions - Own and resolve disagreements around metrics, schemas, and data meaning - Ensure the organization operates from a single, trusted source of truth - Lead and develop a high-performing data team - Manage, mentor, and grow data engineers and analytics engineers - Set expectations for quality, ownership, and execution - Drive accountability and elevate team performance - Ensure platform reliability and data integrity at scale - Establish standards for data quality, testing, and observability - Oversee systems that guarantee consistency, traceability, and correctness - Increase organizational velocity through better data systems - Remove bottlenecks in how data is produced and consumed - Leverage AI tools like OpenAI Codex, Cursor, and Claude to accelerate both individual and team output - Set the standard for how AI is used across the data organization Skills you will have fine-tuned: - Technical leadership and architectural decision-making - Designing end-to-end data platforms that scale with business growth - Balancing tradeoffs across performance, cost, and maintainability - Organizational data alignment - Driving consensus across teams with competing priorities - Defining metrics that hold up under scrutiny and edge cases - Data governance and definition ownership - Establishing clear ownership of metrics, schemas, and contracts - Resolving ambiguity and enforcing consistency across the organization - People leadership and team development - Coaching engineers at different levels of seniority - Building a culture of accountability, ownership, and high standards - Operating reliable data systems at scale - Setting standards for data quality, observability, and system health - Leading incident response and continuous improvement efforts - AI-enabled team acceleration - Scaling the use of AI tools across a team, not just individually - Identifying where AI meaningfully improves speed, quality, and leverage - Building workflows that combine human judgment with AI efficiency What's in it for you: - Opportunity to be an innovator, challenge the status quo, and redefine the payments category - Competitive salary in a fast-growing start-up - Medical insurance with coverage for dependents (parents, spouse, children) - Rewards & Recognition system - Opportunity for personal and professional growth in a dynamic industry

Worldwide
Job Closed

Data Engineer

CellPoint Digital

What makes CellPoint Digital a leader in the payment landscape isn’t just our technology - it’s our people and how we work together. We’ve built a global community where diverse talents and perspectives unite to create innovative solutions. When you join us, you become part of something bigger: a collaborative culture that crosses borders and disciplines, bringing out the best in every team member to deliver breakthrough results for our clients and partners. Together, we are transforming the payments industry - challenging, supporting, and inspiring one another in the process.

Data Engineer113 days ago

Role Description Join us as a Data Engineer (Payments / Orchestration) on our mission to turn payments into possibilities! You will own the foundation of our payments data platform , ensuring that transaction data is reliable, consistent, and production-grade from ingestion through consumption. This role is responsible for building and maintaining high-quality data pipelines , enforcing data contracts, and ensuring event integrity across the system. This is not a reporting role . You are responsible for making sure the data exists, is correct, and can be trusted at scale. We are a high-velocity team. You are expected to leverage AI tools to accelerate development, analysis, and problem-solving. How You Will Make an Impact: - Build and maintain reliable data pipelines - Design and operate ETL / ELT pipelines that process high-volume transaction data - Ensure data is delivered accurately and on time across all downstream systems - Ensure database performance and reliability - Manage and optimize databases for performance, scalability, and uptime - Prevent bottlenecks and ensure systems can handle production load - Enforce data contracts and schema consistency - Define and enforce schemas across services and pipelines - Ensure event integrity and prevent breaking changes across systems - Improve data reliability and observability - Implement monitoring, alerting, and logging across data systems - Detect and resolve data issues before they impact the business - Enable event traceability and consistency - Ensure events can be traced end-to-end across systems - Support debugging and root cause analysis for data discrepancies - Raise the bar on engineering velocity - Leverage AI tools like OpenAI Codex, Cursor, and Claude (or equivalent) to accelerate development and debugging - Improve speed of delivery without compromising reliability Skills you will have fine-tuned: - Designing and operating production-grade data pipelines - Building scalable ETL / ELT systems for high-throughput event data - Managing dependencies, failures, and recovery strategies - Database performance and system reliability - Tuning databases for performance and cost efficiency - Designing systems that remain stable under load - Schema design and data contract enforcement - Creating robust schemas that evolve safely over time - Preventing downstream breakages through strong contract discipline - Data observability and incident response - Implementing monitoring systems that surface issues early - Debugging complex, distributed data failures - Event-driven architecture and data consistency - Ensuring consistency across distributed systems - Building traceability into event flows for auditability and debugging - AI-accelerated engineering workflows - Using AI tools to generate, review, and optimize pipeline code - Improving debugging speed and system understanding with AI assistance - Continuously refining how AI is used to increase output and code quality What's in it for you: - Opportunity to be an innovator, challenge the status quo, and redefine the payments category - Competitive salary in a fast-growing start-up - Medical insurance with coverage for dependents (parents, spouse, children) - Rewards & Recognition system - Opportunity for personal and professional growth in a dynamic industry Company Description What makes CellPoint Digital a leader in the payment landscape isn’t just our technology - it’s our people and how we work together. We’ve built a global community where diverse talents and perspectives unite to create innovative solutions. When you join us, you become part of something bigger: a collaborative culture that crosses borders and disciplines, bringing out the best in every team member to deliver breakthrough results for our clients and partners. Together, we are transforming the payments industry - challenging, supporting, and inspiring one another in the process.

Worldwide
Job Closed
Rackspace Technology logo

Intern I - IN

Rackspace Technology

Where enterprise AI runs and outcomes scale

Data Engineer113 days ago
Full TimeRemoteTeam 5,001-10,000Since 1998H1B No Sponsor

Job Title: Intern-Software Engineer Location: Remote Duration: 6 Months (Paid Internship of INR 20,000 (monthly) for final year BE/ BTECH student ) Department: AI Transformation About the Role We are looking for a passionate and driven Software Engineer Intern to join our AI Industrialization team. This is an excellent opportunity to work in a collaborative, fast-paced environment, gaining hands-on experience in building innovative software solutions using AI. Key Responsibilities • Build and maintain full stack features for AI-powered internal applications • Integrate LLM capabilities into web applications using LangChain and AWS Bedrock • Write clean, well-tested Python backend services and APIs • Work with the team to deploy services on AWS and Kubernetes • Collaborate with senior engineers and the tech lead to break down requirements and deliver features • Learn and apply best practices in AI application development, prompt engineering, and cloud infrastructure Qualifications • Final year students of B.E. / B. Tech / MCA/ M. Tech (Computer Science) • Eagerness to learn, adapt, and work on new challenges. • A proactive and positive attitude toward learning and problem-solving. • Strong communication skills and ability to collaborate effectively in a team. • Passion for technology and software development. Must-Have Skills • Python — solid working proficiency for backend development and scripting • Full Stack Development — experience building features with a frontend framework (React or similar) and a backend API • AWS — foundational knowledge of core AWS services (S3, Lambda, EC2 or ECS) • LLM / AI APIs — some hands-on experience calling or integrating LLM APIs (OpenAI, Bedrock, or similar) • REST APIs — ability to build and consume RESTful APIs • Git & CI/CD — comfortable with version control and basic deployment pipelines Good-to-Have Skills • Exposure to LangChain or other LLM orchestration frameworks • Familiarity with Kubernetes or containerized deployments (Docker) • Basic understanding of RAG concepts or vector databases • Experience with AWS Bedrock or other managed AI services • Familiarity with agile development practices • Exposure to prompt engineering basics Why Join Us? • Hands-on experience with real-world projects and cutting-edge technology. • Mentorship and learning opportunities from experienced software engineers. • A collaborative and innovative work environment. • Potential for future full-time employment opportunity About Rackspace Technology We are the multicloud solutions experts. We combine our expertise with the world’s leading technologies — across applications, data and security — to deliver end-to-end solutions. We have a proven record of advising customers based on their business challenges, designing solutions that scale, building and managing those solutions, and optimizing returns into the future. Named a best place to work, year after year according to Fortune, Forbes and Glassdoor, we attract and develop world-class talent. Join us on our mission to embrace technology, empower customers and deliver the future. More on Rackspace Technology Though we’re all different, Rackers thrive through our connection to a central goal: to be a valued member of a winning team on an inspiring mission. We bring our whole selves to work every day. And we embrace the notion that unique perspectives fuel innovation and enable us to best serve our customers and communities around the globe. We welcome you to apply today and want you to know that we are committed to offering equal employment opportunity without regard to age, color, disability, gender reassignment or identity or expression, genetic information, marital or civil partner status, pregnancy or maternity status, military or veteran status, nationality, ethnic or national origin, race, religion or belief, sexual orientation, or any legally protected characteristic. If you have a disability or special need that requires accommodation, please let us know.

India
₹20K / month
Job Closed