CXG logo
CXG

Transforming Together. Transforming Experiences.

Data Engineer – Web Scraping, Contract

Data EngineerData EngineerContractRemoteMid LevelTeam 51-200Since 2006H1B No SponsorCompany SiteLinkedIn

Location

Thailand

Posted

13 days ago

Salary

0

Seniority

Mid Level

Bachelor Degree2 yrs expEnglishAirflowAzureCloudETLNoSQLPythonSQLTerraform

Job Description

Data Engineer – Web Scraping, Contract

CXG

• Maintain and manage website scraping configurations using Python. • Monitor scraping configurations for errors and potential crashes. • Oversee retrieved data to detect potential issues and blockages. • Coordinate with stakeholders to understand scraping task requirements and report issues. • Prepare and share periodic reports on scraping activities with stakeholders. • Develop necessary pipelines to ingest data into the Datalake and perform required transformations.

Job Requirements

  • Proven experience in data engineering with expertise in designing and implementing scalable data architectures.
  • Strong experience with ETL processes, data modeling, and data warehousing (Airflow & DBT preferred).
  • Expertise in database technologies, both relational (SQL) and NoSQL.
  • Knowledge of cloud platforms, particularly Azure.
  • Solid understanding of data security measures and compliance standards.
  • Excellent Python experience for data engineering and automation.
  • Strong collaboration skills to work closely with data scientists and analysts.
  • Ability to optimize data pipelines for performance and efficiency.
  • Ability to build, test, and maintain tasks and projects.
  • Experience with version control systems, such as Git.
  • Hands-on experience with Airflow and/or DBT.
  • Experience with Terraform for infrastructure management.
  • Minimum 2 years of experience in a similar role.
  • Strong academic background in a relevant field.
  • Fluent in English (French is a plus).

Related Categories

Related Job Pages

More Data Engineer Jobs

Interval Group logo

Data Architect – Databricks, Azure

Interval Group

High quality consulting. On demand. Delivered by top professionals.

Data Engineer13 days ago
ContractRemoteTeam 51-200H1B No Sponsor

• Technical leadership and guidance of a team of infrastructure and data engineers • Strategic development and refinement of a forward-looking Data Mesh architecture • Managing data platform scaling and ensuring structural integrity within the cloud ecosystem • Close collaboration with cross-functional teams to support data migrations toward a modern Lakehouse environment

Germany
Accenture Federal Services logo

Data Architect

Accenture Federal Services

We believe in the power of change, harnessed in ways that matter for our country and communities.

Data Engineer13 days ago
Full TimeRemoteTeam 10,001+Since 2017H1B No Sponsor

• Use your expertise to drive data and AI architecture and technology decisions • Provide insights that influence the overall strategy and plan for a scalable data solution or product that meets business needs • Educate customers on how to craft highly scalable, flexible, resilient, secure, and cost-effective architectures • Link technology with measurable business value and outcome • Identify and investigate new tools and technologies • Translate client needs into digestible requirements

District Of Columbia + 1 moreAll locations: District Of Columbia | Washington
$126.3K - $243.1K / year
decircle logo

Data Engineer

decircle

Talent Partner for decentralized organizations and projects that are building Web3.

Data Engineer13 days ago
Full TimeRemoteTeam 1-10Since 2019H1B No Sponsor

• Dive deep into blockchain data (Ethereum and other L1/L2s) and enrich it with insights from various data providers. • Build pipelines and infrastructure to power a variety of analytics and dashboards. • Work across teams, design new features, and take ownership of sourcing, verifying, and maintaining high-quality data.

Germany
Job Closed
Full TimeRemoteTeam 1,001-5,000H1B No Sponsor

• Lead and provide expert support for data collection, data validation, data visualization, and analytics initiatives • Apply disciplined methodologies for the planning, analysis, design, and development of information systems on an enterprise-wide basis or across a business sector • Develop analytical techniques and methodologies to solve complex business and technical problems • Perform strategic systems planning, business information planning, and business analysis • Organize and analyze large volumes of structured and unstructured data sets using data analytical tools • Locate, access, merge, clean, and standardize data from multiple sources, and develop derived metrics • Create and implement data collection and analysis tools using programming languages such as Python, Databricks, SQL, Scala, R, and Java • Design, script, debug, and analyze data engineering solutions • Implement and create machine learning-based tools and processes • Apply distributed and parallel processing technologies (e.g., Spark) to handle big data analytics tasks involving large data volumes • Perform SQL Server data imports from CSV and TXT files • Leverage Excel and Google Suite for data analysis, reporting, and collaboration • Utilize supporting tools and platforms such as Pentaho (data import/transformation), Azure Data Studio, GitHub, and Smartsheet as needed • Document task requirements, work completed, processes, and technical details thoroughly • Communicate effectively with stakeholders across technical and business teams • Operate independently as a subject matter expert in a fast-paced, entrepreneurial environment

United States
Job Closed