Data Engineer
Location
Worldwide
Posted
2 days ago
Salary
0
Seniority
Mid Level
Job Description
Data Engineer
DeepSource Technologies
Role Description As a Data Engineer, He/She will be responsible for designing, building, and maintaining the data architecture and infrastructure required for our organization's data needs. They will also be responsible for developing, testing, and deploying data solutions, ensuring that they meet our organization's needs. Qualifications - Experience working with Python for data analysis and processing. - Proficient in using the Pandas library for data manipulation and analysis. - Experience working with Pyspark for distributed data processing. - Familiar with Databricks for managing big data workloads. - Experience with Azure Synapse Spark for building and managing data pipelines. - Experience with Azure Data Factory for creating and managing data integration workflows. - Proficient in SQL and experienced in optimizing SQL queries for performance. - Experience migrating data from SQL Server to other platforms. - Bonus: Experience with Terraform for Azure. - Knowledge assessed in the Databricks certification: Databricks Certified Data Engineering Associate. - Knowledge assessed in the Databricks certification: Databricks Certified Data Engineering Professional. Company Description
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Responsible for designing, building, and maintaining the data architecture and infrastructure required for our organization's data needs. • Develop, test, and deploy data solutions, ensuring they meet our organization's needs.
Role Description Our client is looking for a Senior AWS/ Databricks Engineer in Sandton. - AWS VPC ownership (Databricks VPC): administer and maintain subnets, route tables, security groups, NACLs, NAT/Internet egress patterns (as applicable), and network segmentation to meet performance and security requirements. - Connectivity to the bank / enterprise network: troubleshoot and support end-to-end connectivity between the Databricks AWS VPC (IRE) and the bank network across cross-region/cross-account boundaries; coordinate and drive changes with the Cloud team where required. - Private access to AWS services: Design, implement, and operate VPC endpoints and related routing/DNS patterns to enable secure access to services such as S3 while reducing reliance on public internet paths. - S3 data access enablement (with security controls): Partner with platform/security teams to ensure Databricks workloads can reliably read/write required S3 data using appropriate IAM roles/policies and encryption controls; support diagnosis of access failures that present as platform incidents. - Operational support & reliability: Provide production support for the platform connectivity layer (incident response, RCA, preventative actions), maintain runbooks and reference diagrams, and implement improvements to reduce repeat incidents. - Cross-team change management: Raise, manage, and chase change requests with the Cloud team for items outside the Databricks VPC boundary; translate technical needs into clear implementation requirements and validate changes end-to-end. Qualifications - 6+ years of industry experience - AWS networking: strong hands-on experience with VPC design/operations, routing, security groups/NACLs, and network troubleshooting in production. - 5+ years in enterprise cloud operations: experience operating within a regulated/enterprise environment with change management, auditability, and strict security controls. - 3+ years in connectivity troubleshooting: ability to diagnose reachability issues across complex boundaries (cross-account/cross-region, enterprise network perimeters) and drive resolution across multiple teams. - 5+ years in AWS service access patterns: experience enabling secure access to services like S3 (and related IAM policy patterns) in a way that supports production workloads. - 3+ years in stakeholder management: proven ability to liaise with a central cloud/network team, raise and drive changes, and communicate clearly during incidents. - Databricks on AWS experience: understanding of Databricks workspace architecture and its connectivity constraints (data plane/control plane concepts, typical network dependencies). - Private connectivity patterns: experience with private endpoint patterns and enterprise connectivity services (e.g., endpoint-based access, centralized routing constructs). - Infrastructure-as-Code: Terraform/CloudFormation experience for repeatable, audited changes (nice-to-have). - Security tooling and monitoring: exposure to logging/monitoring approaches used for network and cloud operations. Ways of Working - Owns outcomes end-to-end (hands-on fixes inside the VPC; drives changes outside the boundary through the Cloud team). - Strong operational mindset prioritizes stability, clear communication, and measurable prevention of repeat incidents. - Documents and standardizes runbooks, network diagrams, and repeatable change patterns.
Senior Education and Child Welfare Data Management and Visualization Specialist
ICFFounded in 1969, ICF is a global advisory and technology services company headquartered in Reston, Virginia. It delivers data-driven solutions across energy, environment, infrastru
Role Description ICF seeks an on-call Senior Education and Child Welfare Data Management and Visualization Specialist to work within our Child and Family Well-Being (CFWB) Line of Business. This position specializes in Power BI and Tableau, connecting varied data sets to inform programmatic insights, data storytelling, and using data to help our clients understand progress towards achieving their goals. The Senior Education and Child Welfare Data Management and Visualization Specialist will apply academic knowledge and work experience to support education and child welfare evaluation and continuous quality improvement (CQI) on research and technical assistance projects in a state. Key Responsibilities - Lead the collection, management, and analysis of data to inform program insights and improve data accessibility for users. - Develop dashboards, reports, infographics, and data-driven stories. - Collect data through surveys, focus groups, and interviews. - Carry out essential administrative data-management tasks. - Help build the skills of data analysts by providing guidance on Power BI, data visualization best practices, and approaches for generating meaningful insights aligned with project needs. - Support, design, and build interactive and static dashboards, reports, and data visualizations using Power BI, Tableau, and/or Excel. - Support quality assurance tasks, building data infrastructure, data management and analysis, and data cleaning and quality control processes. - Design data-driven storytelling using data visualization techniques and best practices. - Maintain and rapidly produce regular reporting deliverables and user-friendly reports/presentations and infographics for a variety of audiences. - Partner with teams to define key performance indicators, business intelligence, and data reporting requirements. - Support and mentor data staff across multiple projects in building skills in Power BI, Tableau, and other data management and visualization tools. - Conduct basic descriptive analyses of data to generate reports and data visuals of program activities, outputs/deliverables, and outcomes. - Conduct occasional observations or visits to locations nationwide. Qualifications - Bachelor's degree in data science, education, sociology, social work, psychology, justice, or other social science related subject. - 4+ years of experience conducting qualitative and/or quantitative data collection or management, research analysis, or evaluation. - Demonstrated experience in using software such as Power BI, Tableau, and Power Query is required. - Demonstrated experience in data visualization and data storytelling techniques and principles is required. - Demonstrated experience in producing high-quality, professional visualizations, presentations, products, and/or resources is required. Preferred Skills/Experience - Master’s degree in data science, education, sociology, social work, psychology, justice, or other social science related subject. - Program certificate, coursework, or training in data visualization or data storytelling. - Strong organizational skills with proven ability to manage multiple tasks and maintain attention to detail. - Experience working independently to develop innovative solutions to client data interests and challenges. - Foundational knowledge of SQL for querying and validating data. - Experience sharing knowledge and building data management or visualization skills of colleagues. - Prior professional work experience in a consulting environment. Professional Skills - Demonstrated ability to take ownership of project tasks and follow through on commitments. - Strong interest and/or experience in areas such as early education; K–12 and higher‑education research and evaluation; refugee, immigrant, and asylee services; child welfare; capacity building; technical assistance; mental and behavioral health; implementation science; and continuous quality improvement (CQI). - Proven ability to prioritize and manage multiple activities in a fast‑paced, evolving environment. - Effective team builder and team player with the ability to navigate competing priorities, perspectives, and stakeholder needs. - Strong analytical, problem‑solving, and decision‑making skills. - Excellent verbal, written, and interpersonal communication skills, including experience developing and delivering presentations and technical writing. Pay Range The pay range for this position based on full-time employment is: $74,090.00 - $125,954.00.
Senior Education and Child Welfare Data Management and Visualization Specialist
ICFFounded in 1969, ICF is a global advisory and technology services company headquartered in Reston, Virginia. It delivers data-driven solutions across energy, environment, infrastru
Role Description ICF seeks an on-call Senior Education and Child Welfare Data Management and Visualization Specialist to work within our Child and Family Well-Being (CFWB) Line of Business. This position specializes in Power BI and Tableau, connecting varied data sets to inform programmatic insights, data storytelling, and using data to help our clients understand progress towards achieving their goals. The Senior Education and Child Welfare Data Management and Visualization Specialist will apply academic knowledge and work experience to support education and child welfare evaluation and continuous quality improvement (CQI) on research and technical assistance projects in a state. Key Responsibilities - Lead the collection, management, and analysis of data to inform program insights and improve data accessibility for users. - Develop dashboards, reports, infographics, and data-driven stories. - Collect data through surveys, focus groups, and interviews. - Carry out essential administrative data-management tasks. - Help build the skills of data analysts by providing guidance on Power BI, data visualization best practices, and approaches for generating meaningful insights aligned with project needs. - Support, design, and build interactive and static dashboards, reports, and data visualizations using Power BI, Tableau, and/or Excel. - Support quality assurance tasks, building data infrastructure, data management and analysis, and data cleaning and quality control processes. - Design data-driven storytelling using data visualization techniques and best practices. - Maintain and rapidly produce regular reporting deliverables and user-friendly reports/presentations and infographics for a variety of audiences. - Partner with teams to define key performance indicators, business intelligence, and data reporting requirements. - Support and mentor data staff across multiple projects in building skills in Power BI, Tableau, and other data management and visualization tools. - Conduct basic descriptive analyses of data to generate reports and data visuals of program activities, outputs/deliverables, and outcomes. - Conduct occasional observations or visits to locations nationwide. Qualifications - Bachelor's degree in data science, education, sociology, social work, psychology, justice, or other social science related subject. - 4+ years of experience conducting qualitative and/or quantitative data collection or management, research analysis, or evaluation. - Demonstrated experience in using software such as Power BI, Tableau, and Power Query is required. - Demonstrated experience in data visualization and data storytelling techniques and principles is required. - Demonstrated experience in producing high-quality, professional visualizations, presentations, products, and/or resources is required. Requirements - Master’s degree in data science, education, sociology, social work, psychology, justice, or other social science related subject (preferred). - Program certificate, coursework, or training in data visualization or data storytelling (preferred). - Strong organizational skills with proven ability to manage multiple tasks and maintain attention to detail. - Experience working independently to develop innovative solutions to client data interests and challenges. - Foundational knowledge of SQL for querying and validating data. - Experience sharing knowledge and building data management or visualization skills of colleagues. - Prior professional work experience in a consulting environment. Professional Skills - Demonstrated ability to take ownership of project tasks and follow through on commitments. - Strong interest and/or experience in areas such as early education; K–12 and higher‑education research and evaluation; refugee, immigrant, and asylee services; child welfare; capacity building; technical assistance; mental and behavioral health; implementation science; and continuous quality improvement (CQI). - Proven ability to prioritize and manage multiple activities in a fast‑paced, evolving environment. - Effective team builder and team player with the ability to navigate competing priorities, perspectives, and stakeholder needs. - Strong analytical, problem‑solving, and decision‑making skills. - Excellent verbal, written, and interpersonal communication skills, including experience developing and delivering presentations and technical writing. Pay Range The pay range for this position based on full-time employment is: $74,090.00 - $125,954.00.

