Software Mind logo
Software Mind

Software House focused on results since 1999

Senior Data Engineer – AWS

Data EngineerData EngineerFull TimeRemoteSeniorTeam 1,001-5,000Since 1999H1B No SponsorCompany SiteLinkedIn

Location

Poland

Posted

80 days ago

Salary

0

Seniority

Senior

Job Description

Senior Data Engineer – AWS

Software Mind

• Develop reusable, metadata-driven data pipelines with strong focus on database design and SQL optimization • Work extensively with AWS-based datasets and data infrastructure including Redshift, RDS, S3, Glue, Athena, and DMS • Support comprehensive ETL development and optimization using advanced SQL techniques and query performance tuning • Automate and optimize data platform processes leveraging AWS data services ecosystem • Build robust integrations with data sources and consumers using AWS native solutions • Proactively resolve performance and data quality issues through SQL optimization and database tuning • Leverage DBT for data transformation modeling and Jenkins/GitHub Actions for CI/CD automation • Contribute to platform documentation and runbooks with focus on database best practices • Propose and implement improvements to data platform architecture using AWS data services

Job Requirements

  • Strong database expertise: advanced SQL skills, comprehensive data modeling experience, and proven query optimization capabilities
  • AWS data services proficiency: hands-on experience with Redshift, RDS, S3, Glue, Athena, and DMS
  • ETL development: strong background in ETL development and optimization using database-centric approaches
  • Programming: Python/PySpark and advanced SQL for data pipeline development
  • Spark proficiency: expert-level skills in building data pipelines using Databricks on AWS or EMR/EMR Serverless
  • Large-scale data processing: extensive experience working with complex datasets and database performance optimization
  • Library development: skilled in building reusable data transformation modules as Python packages
  • CI/CD expertise: experience with GitHub Actions, Jenkins, and AWS CodePipeline for automated deployments
  • AWS networking/security: Understanding of VPC, subnets, security groups, and IAM
  • Agile methodology: familiar with Scrum development practices

Benefits

  • Flexible employment and remote work
  • International projects with leading global clients
  • International business trips
  • Non-corporate atmosphere
  • Language classes
  • Internal & external training
  • Private healthcare and insurance
  • Multisport card
  • Well-being initiatives

Related Categories

Related Job Pages

More Data Engineer Jobs

TOPMIND logo

Mid-level Data Cloud Developer – Data Engineer

TOPMIND

Há 21 anos no mercado, com atuação Latam e Estados Unidos, a TOPMIND é umas das principais empresas de TI.

Data Engineer80 days ago
Full TimeRemoteTeam 501-1,000Since 1994H1B No Sponsor

• Develop and maintain data pipelines in a cloud environment. • Model data for consumption by reports, APIs, and data products. • Implement best practices for data quality, versioning, and data testing. • Optimize query performance and data loading processes. • Collaborate with product, BI, and engineering teams to understand requirements. • Monitor systems, handle incidents, and ensure solution reliability. • Document processes, data flows, and data standards.

Brazil
Scotch logo

Senior Backend Engineer - Data Onboarding

Scotch

From neighborhood stores to destination warehouses, liquor retailers of all sizes have been held back by manual tasks, slow systems and limited business intelligence. So we decided to build something better — smarter software that saves time and improves margins so independent retailers could focus on their customers, not manually managing inventory.

Data Engineer80 days ago
Full TimeRemoteTeam 25Since 2024

Senior Software Engineer, Data @ Scotch About Us Scotch’s mission is to modernize one of the oldest industries on earth: the $1.6T beverage alcohol market. We’re starting by building the next-generation operating system for BevAlc retailers (POS, payments, Back Office, E-commerce). We are seeking a detail-oriented Senior Backend Engineer to join our data onboarding team. You will be a critical component of our team that builds out the tools and automations needed to onboard new retailers at scale. You’ll deliver key components and functionality needed to ensure data quality, accuracy and reliability while working with a variety of data structures and challenges. Key Responsibilities We’re a lean and efficient team but growing quickly, and this role is critical to the next phase of our business. Your core responsibilities include: - Data Pipeline: You will help extract data from legacy POS systems, serving as the bridge between old technology and our modern platform. - Automation: You will build tools to transform and validate data to ensure all fields meet requirements and accuracy standards before importing into Scotch - Process Optimization: You will find ways through automation, AI, and other tools to streamline and optimize the onboarding process - Collaboration: You will use your strong communication skills to work with our onboarding team to most efficiently bring on new customers What We Value For this engineering role, we specifically value: - Technical Execution: A core strength in getting things done technically and effectively. - Organizational Rigor: A high level of attention to detail and the ability to manage a pipeline of tasks simultaneously without dropping the ball. - Independence: The ability to work autonomously to solve problems and move tasks forward. - Customer Empathy: Our customers trust us to help manage their business, so we must do what we can to help them. Who You Are - 5+ years of relevant professional experience in software development or data engineering (We use Ruby on Rails and Postgres as our backend tech stack) - Hands-on experience with production application systems and data pipelines - Expert knowledge of performance optimization techniques including query optimization, data transformation, and async job processing - Demonstrated organizational rigor and attention to detail. - Strong cross-functional collaborator across engineering, product, and operations teams. - Clear communicator who drives alignment and moves projects forward quickly. Bonus Points - Experience specifically in the POS or fintech industry. - Experience in the alcohol industry - Experience working with AI tools and incorporating them into workflows - Proficiency as a full stack engineer or front end frameworks is a nice to have - Familiarity with legacy data extraction and cleaning processes. Compensation & Benefits - Competitive salary and equity - Health, dental, and vision - Company sponsored 401k - Unlimited, flexible PTO - Bi-Annual Company Offsites — company-paid trips to places like Jackson Hole, Park City, Lake Tahoe, and more to work alongside & bond with your colleagues. Equal Opportunity Employer We are an equal-opportunity employer. We don’t discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We also consider for employment qualified applicants with arrest and conviction records, consistent with applicable federal, state and local law, including but not limited to the San Francisco Fair Chance Ordinance. Questions When Applying - What has you interested in this opportunity? - Where are you currently located or will you be based if offered this position? - What is your stance on AI productivity tools and AI workflows?

United States
Job Closed
Optum logo

Senior Big Data Software Engineer - Remote

Optum

Optum, part of the UnitedHealth Group family of businesses, is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together. At Optum, we support your well-being with an understanding team, extensive benefits and rewarding opportunities. By joining us, you’ll have the resources to drive system transformation while we help you take care of your future. We recognize the power of connection to drive change, improve efficiency and make a difference in health care. Join a team where your skills and ideas can make an impact and where collaboration is key to creating technology that produces healthier outcomes.

Data Engineer80 days ago
Full TimeRemoteTeam 160,000Since 2011

Requisition Number: 2318677 Optum Insight is improving the flow of health data and information to create a more connected system. We remove friction and drive alignment between care providers and payers, and ultimately consumers. Our deep expertise in the industry and innovative technology empower us to help organizations reduce costs while improving risk management, quality and revenue growth. Ready to help us deliver results that improve lives? Join us to start Caring. Connecting. Growing together. You'll enjoy the flexibility to work remotely * from anywhere within the U.S. as you take on some tough challenges. For all hires in the Minneapolis or Washington, D.C. area, you will be required to work in the office a minimum of four days per week. Primary Responsibilities: - Design, code, test, document, and maintain high-quality and scalable big data solutions - Research, evaluate, and deploy new tools, frameworks and patterns to help build a sustainable big data platform - Identify gaps and opportunities for improvement of existing solutions - Define and develop APIs for integration with various data sources in the enterprise - Translate customer requirements into solution design - Make accurate development effort estimates to assist management in project and resource planning - Collaborate with management, quality assurance, architecture, and other development teams - Write technical documentation and participate in production support - Participate with prototypes as well as design and code reviews - Keep skills up to date through ongoing self-directed training The ideal candidate will be a self-starter who can learn things quickly, who is enthusiastic and actively engaged You'll be rewarded and recognized for your performance in an environment that will challenge you and give you clear direction on what it takes to succeed in your role as well as provide development for other roles you may be interested in. Required Qualifications: - Undergraduate degree or equivalent experience - 6+ years of hands-on software development experience - 5+ years of development experience with Java or Scala, Python, JSON, Web Services - 2+ years of experience in distributed systems technologies like Spark, Databricks, Hadoop, Snowflake - 3+ years of experience in relational databases (SQL) - Experience with Agile/SCRUM methodology Preferred Qualifications: - Experience in infrastructure services on at least one major cloud platform (Azure preferred) - Experience developing rules in rules engine - 2+ years of experience in Kafka streaming - 2+ years with NoSQL databases (Cassandra preferred) - 2+ years working with Docker and Kubernetes - Thorough understanding of service-oriented architecture (SOA) concepts - Experience defining and developing APIs for integration with various data sources in the enterprise *All employees working remotely will be required to adhere to UnitedHealth Group's Telecommuter Policy Pay is based on several factors including but not limited to local labor markets, education, work experience, certifications, etc. In addition to your salary, we offer benefits such as, a comprehensive benefits package, incentive and recognition programs, equity stock purchase and 401k contribution (all benefits are subject to eligibility requirements). No matter where or when you begin a career with us, you'll find a far-reaching choice of benefits and incentives. The salary for this role will range from $xx,xxx to $xx,xxx annually based on full-time employment. We comply with all minimum wage laws as applicable. Application Deadline: This will be posted for a minimum of 2 business days or until a sufficient candidate pool has been collected. Job posting may come down early due to volume of applicants. At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission. UnitedHealth Group is an Equal Employment Opportunity employer under applicable law and qualified applicants will receive consideration for employment without regard to race, national origin, religion, age, color, sex, sexual orientation, gender identity, disability, or protected veteran status, or any other characteristic protected by local, state, or federal laws, rules, or regulations. UnitedHealth Group is a drug - free workplace. Candidates are required to pass a drug test before beginning employment.

Minnesota
Job Closed

AI Data Engineer

Netwrix

Netwrix is a cybersecurity company specializing in data security solutions that help organizations identify and protect sensitive information, ensuring regulato

Data Engineer80 days ago

• Design, build and operate data and AI platforms as code, using Git‑based workflows as the source of truth. • Implement pull‑request‑driven change control, automated testing and CI/CD pipelines. • Define and maintain Infrastructure‑as‑Code for data and AI systems to ensure consistency, traceability, and rollback capability. • Design and maintain scalable ETL/ELT pipelines that support AI/ML model training and retraining, feature engineering and feature stores, batch and near‑real‑time inference workflows. • Support Retrieval‑Augmented Generation (RAG) and internal AI agents by curating, indexing and refreshing select data sources. • Implement proactive monitoring for data quality and schema integrity, pipeline performance and failure modes. • Partner with Product and Engineering teams to operationalize models by integrating data pipelines into MLOps workflows. • Build and operate Azure-based data and AI platforms, including storage, compute, orchestration and containerized services.

United States
Job Closed