Azure Data Engineer

Platform EngineerPlatform EngineerFull TimeRemoteMid LevelTeam 201-500

Location

Latin America (LATAM)

Posted

3 days ago

Salary

0

Seniority

Mid Level

Job Description

Azure Data Engineer

Bluelight Consulting

Role Description We are looking for a skilled individual to join our rapidly growing team at Bluelight. This position is ideal for someone who thrives in a fast-paced, dynamic environment where everyone's opinions and efforts are valued and appreciated. You will have the opportunity to contribute to challenging and meaningful projects, developing high-quality applications that stand out in the market. We value continuous learning, personal growth, and hard work, offering a collaborative environment that promotes professional development. If you are passionate about software development and eager to be part of a growing software consultancy, we invite you to apply and join us on this exciting journey. - ETL Data Engineering: Develop and maintain ETL data engineering processes using Python (PySpark) within Azure Synapse Analytics Notebooks, and/or Azure Synapse Analytics Pipelines, to ensure efficient data extractions, transformation, and loading. - Data Warehousing: Apply your expertise in data warehousing, understanding star schemas, facts, and dimensions, to design and build effective data storage structures in a Massively Parallel Processing (MPP) SWL Pool. - Data Source Expertise: Extract data from various sources, including REST APIs, SWL database tables, and CSV files. - Azure Synapse Analytics Expertise: Utilize your deep knowledge of Azure Synapse Analytics to design and optimize data notebooks/pipelines for scalability and performance. - Data Fabric Concepts: Contribute to the implementation and understanding of other Data Fabric concepts, such as data lakes, lakehouses, delta lakes, and data cataloging, to enhance data management capabilities. - Data Modeling: Collaborate with data architects to create data models and schemas that align with business requirements. - Data Quality: Implement data quality checks and validation processes to maintain data accuracy and consistency. - Performance Tuning: Identify and resolve performance bottlenecks and optimize ETL data notebooks/pipelines to meet SLAs. - Monitoring and Troubleshooting: Monitor ETL jobs, diagnose issues, and implement solutions to ensure data pipeline reliability. - Documentation: Maintain comprehensive documentation of ETL data engineering processes, data flows, and data transformations. - Collaboration: Work closely with cross-functional teams to understand data requirements and provide support for data-related initiatives. - Security and Compliance: Ensure data security and compliance with data governance and privacy standards. Qualifications - Bachelor’s degree in Computer Science, Information Technology, or a related field; or equivalent work experience, with certifications related to data engineering or data science (e.g. Azure Data Engineer) being a plus. - Proven experience in ETL data engineering with significant expertise in using Python (PySpark) to perform data extraction, transformation, and loading from REST APIs, SQL database tables, and CSV files. - Proficiency in using Azure Synapse Analytics resources including Notebooks, Pipelines, Linked Services, and Azure Key Vault. - Demonstrated ability to write complex SQL queries, optimize query performance, and work with both SparkSQL and MS SQL to effectively extract, transform, and load data. - Knowledge of data integration best practices and tools. - Experience with version control systems, such as Git (Azure DevOps). - Strong problem-solving and analytical skills, with a keen attention to detail. - Excellent communication skills, both verbal and written, with the ability to work collaboratively in a team environment with shifting priorities. - Familiarity with big data technologies, machine learning, and data analysis preferred. - Experience with data visualization tools (e.g. Power BI, Tableau) and Agile Methodologies a plus. Benefits - Competitive salary and bonuses, including performance-based salary increases. - Generous paid-time-off policy. - Flexible working hours. - Work remotely. - Continuing education, training, conferences. - Company-sponsored coursework, exams, and certifications.

Related Categories

Related Job Pages

More Platform Engineer Jobs

OneDome logo

Senior Platform Engineer

OneDome

Faster, simpler homebuying. Guaranteed.

Full TimeRemoteTeam 51-200H1B No Sponsor

• Own 5 AWS accounts across the organisation (QA, Prod, Infra, plus two others) • Architect and maintain infrastructure as code with Terraform. • Replace the click-ops that's still around • Stand up CI/CD pipelines engineers actually want to use. Blue/green and canary where it makes sense • Run releases reliably and reduce key-person risk on the deploy process • Set up monitoring, alerting, and incident response so we catch issues before customers do. • Define SLIs and SLOs that map to real user journeys • Lead incident response as the primary infra responder. Run blameless post-mortems • Patch and harden the platform. We're a fintech, this matters. IAM least privilege, secrets management, vulnerability scanning, MFA enforcement • Lead infrastructure work for the TV launch: traffic spikes, autoscaling, capacity planning, runbooks • Partner with backend engineers when production issues cross into the app tier. JVM tuning, connection pools, async patterns, memory leaks • Architect multi-region DR. • Drive the platform toward self-healing • Optimise cost across DynamoDB, Lambda, and the wider AWS footprint • Set architectural guardrails with tech leads and SWEs. • Mentor engineers as the platform capability grows.

Azerbaijan
$4K - $5K / month
1Password logo

Senior Data Platform Engineer

1Password

Productive businesses use 1Password to secure employees at scale.

Full TimeRemoteTeam 501-1,000Since 2009H1B Sponsor

• Help build the next generation of our data platform, a streaming-first, domain-oriented foundation • Work on infrastructure, tooling, and paved paths for data flow from production systems • Enable near real-time access for analysts, engineers, AI systems, and business users • Champion platform standards, tooling, and best practices across the data organization

California
$152K - $197.5K / year
SH logo

Platform Engineer

SH

Leader in Formwork, Scaffolding, and Shoring. SH, The Support Your Project Needs!

Full TimeRemoteTeam 501-1,000Since 1969H1B Sponsor

• Partner with product engineering teams to identify systems requirements. • Build and support our cloud-based microservices infrastructure. • Automate routine processes and remediation tasks. • Develop, monitor and track Service Level Objectives (SLOs) for the systems under management. • Proactively troubleshoot, resolve, and plan for issues that typically come from support staff, other engineering teams, and our automated monitoring system. • Ensure our datastores are healthy and operate at optimal performance levels. • Contribute to the growth and culture of our engineering team.

France
Referrals Only logo

Lead Platform Engineer

Referrals Only

Thoughtworks is a dynamic and inclusive community of bright and supportive colleagues who are revolutionizing tech. As a leading technology consultancy, we’re pushing boundaries through our purposeful and impactful work. For 30+ years, we’ve delivered extraordinary impact together with our clients by helping them solve complex business problems with technology as the differentiator. Bring your brilliant expertise and commitment for continuous learning to Thoughtworks. Together, let’s be extraordinary.

Full TimeRemoteTeam 11-50

Role Description Lead Platform Engineers help clients build and evolve systems that client organizations use to deliver and run software. They are comfortable working within teams of people with diverse roles and levels of experience to find solutions that meet the needs of the organization. They combine technical expertise and understanding with consideration of different situational needs. They champion technical quality and effective ways of working as a means to better outcomes for clients, rather than an end in themselves. They help clients to understand agile ways of working and DevOps as a mindset for collaboration. Job Responsibilities - Explore the client’s needs and drive the building of a technical roadmap and impactful solution that will support their ambitious business goals. - Help shape and build Thoughtworks’ cloud and infrastructure practice through collaboration with business development, marketing, and capabilities development teams. - Ensure and build the controls and processes for continuous delivery and evolution of infrastructure and applications, driving automation through all stages of the process. - Take a leading role in monitoring and ensuring that technical expectations of deliverables are consistently met on projects. - Provide expertise and guidance in the areas of DevOps, cloud, platform and infrastructure engineering, both internally and in client sites. - Establish trusting and thoughtful partnerships with a client’s leadership across engineering and commercial functions. - Lead the design and implementation of innovative solutions to current constraints and business policies. Qualifications - Lead the collaborative design of enterprise and/or web-scale hosting platforms and can administer application servers, web servers and databases. - Deep understanding of cloud and virtualization platforms, infrastructure automation and application hosting technologies. - Experience working with software delivery teams and understand DevOps philosophies, Agile methods, Infrastructure as Code and how to apply them to your work. - Regularly apply DevOps philosophy, Agile methods, Infrastructure as Code to your work and lead infrastructure and operations with these approaches. - History of working with server virtualization, at least one IaaS cloud platform, and two or more application runtime platforms including physical servers, virtual servers, container clusters, serverless and databases. - Ability to write scripts using at least one scripting language and comfortable with building one or more of: Linux servers, Windows servers or container clusters, and/or Windows servers systems. - Experience with continuous integration and continuous delivery tools with different tech stacks, web or mobile. - Previously worked with monitoring systems for availability, performance or security, stress and performance testing with observability patterns. - Understanding of security concerns, threats and approaches for dealing with them, including infrastructure platform vulnerabilities, secrets management, network security and software supply chain security. - Bonus points for experience with unit testing and automated testing tools, stress and performance testing. Requirements - Genuinely enjoy interacting with teammates from across the business and have a knack for communicating technical concepts to nontechnical audiences. - Love creating robust, scalable, flexible and relevant solutions that help transform businesses and industries. - Comfortable partnering directly with leadership teams (CTO/CIO/COO) to design technical strategies while simultaneously collaborating with senior IT groups in an advisory capacity. - Adapt effortlessly to uncertainty, embrace change and confidently make decisions with limited information to achieve positive outcomes. Benefits - No one-size-fits-all career path at Thoughtworks; career development is entirely up to you. - Balance autonomy with the strength of a cultivation culture supported by interactive tools and numerous development programs. - Value in helping each other be our best and empowering employees in their career journeys. Company Description Thoughtworks is a dynamic and inclusive community of bright and supportive colleagues who are revolutionizing tech. As a leading technology consultancy, we’re pushing boundaries through our purposeful and impactful work. For 30+ years, we’ve delivered extraordinary impact together with our clients by helping them solve complex business problems with technology as the differentiator. Bring your brilliant expertise and commitment for continuous learning to Thoughtworks. Together, let’s be extraordinary.

Ecuador