Job Closed
This listing is no longer active.
As a pioneer for digital transformation GFT develops sustainable solutions across new technologies.
Data Engineer (Mid/Senior)
Location
Brazil
Posted
68 days ago
Salary
0
Seniority
Senior
Job Description
Data Engineer (Mid/Senior)
GFT Technologies
• Expertly develop backend components in Python; • Design and build scalable data pipelines using AWS services such as S3, Glue, Athena, Lambda, and Step Functions; • Implement data ingestion, transformation, and modeling solutions with a focus on high performance and security; • Collaborate with business and data science teams to translate requirements into efficient technical solutions; • Monitor, optimize, and ensure data quality across pipelines and systems; • Apply DevOps practices to data solutions, leveraging CI/CD and automation; • Ensure compliance with regulations and data governance best practices.
Job Requirements
- Required: experience with S3 buckets containing Parquet files;
- Required: knowledge of PySpark, SparkSQL, and AWS Glue;
- Advanced experience with AWS and its core data engineering services (S3, Glue, Lambda, DynamoDB, Athena, etc.);
- Proficiency in programming languages such as Python for data manipulation, processing, and backend development;
- Knowledge of orchestration tools such as Step Functions;
- Experience with relational and non-relational databases (PostgreSQL, MongoDB, DynamoDB);
- Familiarity with AWS security best practices (IAM, KMS, data encryption);
- Experience with data streaming solutions (Kafka, Kinesis);
- Experience with version control and CI/CD pipelines using tools like GitHub, Jenkins, or CodePipeline.
Benefits
- Multi-benefits card — choose how and where to use it.
- Tuition assistance for undergraduate, graduate, MBA, and language courses.
- Certification incentive programs.
- Flexible working hours.
- Competitive salaries.
- Annual performance review with a structured career plan.
- Opportunity for international career development.
- Wellhub and TotalPass.
- Private pension plan.
- Childcare assistance.
- Health insurance.
- Dental insurance.
- Life insurance.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Investigative Data Engineer
Canada's National ObserverIntelligent. Accurate. Journalism for a changing climate
• Develop and maintain features for our flagship application Civic Searchlight for investigative journalism and other use cases • Discover new data sources and build pipelines to integrate them with existing infrastructure • Implement rigorous monitoring and testing procedures so the database stays reliable as coverage grows • Help build new tools on top of the database, including improved search, alerts, automated summaries and AI-assisted features • Contribute to architectural decisions as we move from prototype-scale to production infrastructure
• Serve as a shared resource across product teams to provide expertise and scalable solutions for data integration, modeling, and analytics across all products. • Design and implement scalable data pipelines using AWS services such as AWS Glue, AWS Data Pipeline, and Amazon Kinesis. • Create and maintain all Infrastructure-as-Code (IaC) for primary product teams using tools like AWS CloudFormation or CDK. • Develop and maintain data storage solutions using Amazon S3, Redshift, RDS, DynamoDB, or other AWS-managed databases. • Design and develop code pipelines for our primary product application to: • Build application components. • Run automated tests created by the QA team. • Execute static code analysis. • Allow or prevent commits or PR merges based on predefined quality thresholds. • Deploy and manage AWS infrastructure resources as needed to support organizational and product-specific goals. • Implement monitoring, logging, and centralization of application instrumentation for real-time insights and troubleshooting. • Optimize ETL workflows to efficiently manage data across diverse sources and destinations. • Set up and manage AWS IAM policies, roles, and security-related configurations to ensure secure access and data protection. • Stay updated on AWS innovations and recommend tools or best practices to enhance the organization’s data ecosystem. • Create comprehensive documentation and provide training for AWS-based data solutions, ensuring knowledge transfer and ease of use. • Ensure all data processes and systems adhere to security best practices, aligning with OWASP Top 10, CWE Top 25 guidelines, and CIS AWS Foundations Benchmark.
Data Engineer
Per ScholasPer Scholas is a national organization that has been advancing economic mobility for 25 years. Through rigorous training, professional development, and robust employer connections, we prepare individuals traditionally underrepresented in technology for high-growth careers in the industry.
ABOUT PER SCHOLAS: For 30 years, Per Scholas has been on a mission to drive mobility and opportunity in the ever-advancing technology landscape by unlocking the untapped potential of individuals, uplifting communities, and meeting the needs of employers through rigorous tech training. By teaming up with dynamic employer partners, ranging from Fortune 500 companies to innovative startups, we're forging inclusive tech talent pipelines, fulfilling an ever-increasing need for skilled talent. With national remote training and campuses in 20+ cities and counting, Per Scholas offers no-cost training programs in the most sought-after tech skills, spanning Cloud, Cybersecurity, Data Engineering, IT Support, Software Engineering, and more. To date, 30,000+ individuals have been trained through Per Scholas, propelling their professional trajectories into high-growth tech careers with salaries three times higher than their pre-training earnings. Learn more by visiting PerScholas.org and follow us on LinkedIn, X, Facebook, Instagram, and YouTube. Per Scholas preferred hires reside within the following states: AZ, CA, CO, FL, GA, IL, IN, KS, MD, MA, MI, MO, NC, NJ, NY, OH, PA, TX, WA Position Title: Data Engineer Location: Remote Reports To: EVP, IT Position Summary Per Scholas is seeking a Data Engineer to maintain and expand our enterprise data infrastructure. This role focuses on the technical execution of our data strategy, specifically managing the flow of data into our newly established Snowflake Data Lake. You will be the "bridge" that ensures data from various sources is accurately ingested, transformed, and ready for visualization in Tableau. Core Responsibilities 1. Data Pipeline Management - Set up and monitor automated data ingestion connectors using Fivetran. - Troubleshoot sync errors and work with API providers to ensure continuous data flow. - Write and optimize SQL transformations within Snowflake to prepare data for reporting. - Familiarity or experience with OpenFlow data ingestion is nice-to-have 2. Data Lake Maintenance - Maintain the structure of the Snowflake Data Lake (schemas, tables, and views) as defined in partnership with InterWorks. - Perform regular data quality checks to ensure the "source of truth" remains accurate. - Monitor warehouse performance and assist in managing Snowflake credit consumption. 3. Reporting & Visualization Support - Build and maintain "Published Data Sources" in Tableau for use by various departments. - Assist team members in building complex Tableau dashboards by providing the necessary back-end data structures. - Translate business requests into technical requirements for new data pipelines. 4. Privacy & Compliance - Execute data masking and access control policies to ensure compliance with Data Privacy frameworks. - Assist in maintaining documentation for data lineage and the organizational data dictionary. Required Qualifications - Excellent communication skills, able to communicate technical requirements to non technical audiences - A clear understanding of data modeling and best practices - 2–4 years of professional experience in data engineering or a highly technical data analyst role. - Experience with CI/CD pipelines - Snowflake: Hands-on experience querying and managing tables within a Snowflake environment. - Fivetran: Proficiency in setting up and managing SaaS connectors. - Experience writing and implementing data transformations with dbt (Data Build Tool) - Tableau: Experience in data modeling for Tableau (calculating fields, joining data sets, and optimizing for performance). - Technical Skills: Strong SQL skills are mandatory. Experience with Python for basic scripting is preferred. - Mission-Driven: A genuine interest in using technology to support Per Scholas’ mission of closing the opportunity gap. LI#- Remote For this position, our target salary is $139,000, with a range of $129,000 to $139,000. The variation in compensation typically hinges on factors such as the candidate's geographic location and their qualifications and experience that align with the role. QUESTIONS? If you have any questions about this role, please feel free to email our Talent team at jobs@perscholas.org. We look forward to viewing your application! Equal Employment Opportunity We're proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, race, color, religion, sex, sexual orientation, gender identity or national origin. PII Policies Non-Discrimination Policy
• Develop and continuously improve a data service that underpins cloud-based applications • Support data and database modeling efforts • Contribute to the development and maintenance of reusable component libraries and shared codebase • Participate in the entire software development lifecycle, including requirement gathering, design, development, testing, and deployment, using an agile, iterative process • Collaborate with developers, designers, testers, project managers, product owners, and project sponsors to integrate the data service to end user applications • Communicate tasking estimation and progress regularly to a development lead and product owner through appropriate tools • Ensure seamless integration between database and messaging systems and the frontend / UI they support • Ensure data quality, reliability, and performance through code reviews and effective testing strategies • Write high-quality code, applying best practices, coding standards, and design patterns • Team with other developers, fostering a culture of continuous learning and professional growth



