NIR-YU logo
NIR-YU

Take Control of Your Business and Execute Your Vision with Ease - Hire Affordable and Qualified Nearshore Staff

Data Engineer – Web Scraping, LLM Pipelines, Scalable Data Infrastructure

Infrastructure EngineerInfrastructure EngineerFull TimeRemoteSeniorTeam 201-500H1B No SponsorCompany SiteLinkedIn

Location

Argentina

Posted

8 days ago

Salary

0

Seniority

Senior

Job Description

Data Engineer – Web Scraping, LLM Pipelines, Scalable Data Infrastructure

NIR-YU

• Build new structured datasets, including scraping accelerators, Form D filings and dynamic web sources. • Develop automated ETL pipelines that parse, clean and transform content using LLMs. • Define and maintain database schemas in Supabase or PostgreSQL. • Create evaluation frameworks to measure and compare LLM performance across pipeline components. • Contribute to the design of scalable data architectures using GCP services. • Improve reliability, observability and deployment workflows for scraping and data processing systems.

Job Requirements

  • 4+ years of experience building data pipelines, backend services and automated data processing systems.
  • Strong background in web scraping with tools like Scrapy, Playwright or similar.
  • Experience deploying pipelines on cloud platforms such as GCP or AWS.
  • Solid knowledge of ETL frameworks, workflow orchestration (Airflow) and modern data stores (BigQuery, PostgreSQL).
  • Comfortable working with Docker and API frameworks like FastAPI.
  • Clear, fluent communication in English.

Related Categories

Related Job Pages

More Infrastructure Engineer Jobs

Mirantis logo

AI Infrastructure & Platform Operations Engineer

Mirantis

Strategic open source infrastructure for containers and virtual machines.

Full TimeRemoteTeam 501-1,000H1B Sponsor

• Monitor, operate, and support production AI infrastructure platforms. • Investigate and resolve infrastructure, networking, hardware, and platform-related incidents. • Support NVIDIA GPU infrastructure and associated platform services. • Monitor and troubleshoot Kubernetes-based environments. • Investigate performance, availability, and reliability issues across infrastructure and platform components. • Collaborate with engineering teams, hardware vendors, datacenter personnel, and service delivery teams to resolve technical issues. • Participate in incident response, root cause analysis, and operational improvement activities. • Contribute to improvements in monitoring, observability, automation, and operational processes. • Maintain operational documentation, runbooks, and knowledge articles.

Europe
$60K - $67K / year
Mirantis logo

Senior AI Infrastructure & Platform Operations Engineer

Mirantis

Strategic open source infrastructure for containers and virtual machines.

Full TimeRemoteTeam 501-1,000H1B Sponsor

• Lead the investigation and resolution of complex infrastructure, networking, and platform-related incidents. • Act as a senior escalation point for operational teams during critical service-impacting events. • Support large-scale NVIDIA GPU infrastructure and high-performance networking environments. • Troubleshoot complex Linux, Kubernetes, networking, storage, and hardware-related issues. • Analyze platform performance, capacity, stability, and reliability trends to proactively identify risks. • Lead root cause analysis activities and drive long-term corrective actions. • Collaborate with engineering teams, hardware vendors, and datacenter personnel to resolve complex technical challenges. • Participate in major incident management and service restoration activities. • Provide technical leadership for Kubernetes platform operations and supporting infrastructure services. • Drive improvements in platform reliability, observability, monitoring, and operational processes. • Identify opportunities to automate repetitive operational activities and improve operational efficiency. • Contribute to operational readiness reviews, infrastructure changes, upgrades, and service introductions. • Support the adoption and operation of AI-powered infrastructure services and operational capabilities through k0rdent AI. • Evaluate emerging technologies and operational practices to improve service delivery and platform resilience. • Mentor and support AI Infrastructure & Platform Operations Engineers. • Share technical knowledge through documentation, training sessions, and operational reviews. • Develop and maintain operational standards, runbooks, troubleshooting guides, and best practices. • Help define operational processes, escalation paths, and service reliability standards. • Act as a trusted technical advisor during operational planning and service improvement initiatives.

Europe
$90K - $105K / year
Mirantis logo

AI Infrastructure & Platform Operations Engineer

Mirantis

Strategic open source infrastructure for containers and virtual machines.

Full TimeRemoteTeam 501-1,000H1B Sponsor

• Monitor, operate, and support production AI infrastructure platforms. • Investigate and resolve infrastructure, networking, hardware, and platform-related incidents. • Support NVIDIA GPU infrastructure and associated platform services. • Monitor and troubleshoot Kubernetes-based environments. • Investigate performance, availability, and reliability issues across infrastructure and platform components. • Collaborate with engineering teams, hardware vendors, datacenter personnel, and service delivery teams to resolve technical issues. • Participate in incident response, root cause analysis, and operational improvement activities. • Contribute to improvements in monitoring, observability, automation, and operational processes. • Maintain operational documentation, runbooks, and knowledge articles.

Europe
Mirantis logo

AI Infrastructure, Platform Operations Engineer

Mirantis

Strategic open source infrastructure for containers and virtual machines.

Full TimeRemoteTeam 501-1,000H1B Sponsor

• Monitor, operate, and support production AI infrastructure platforms. • Investigate and resolve infrastructure, networking, hardware, and platform-related incidents. • Support NVIDIA GPU infrastructure and associated platform services. • Monitor and troubleshoot Kubernetes-based environments. • Investigate performance, availability, and reliability issues across infrastructure and platform components. • Collaborate with engineering teams, hardware vendors, datacenter personnel, and service delivery teams to resolve technical issues. • Participate in incident response, root cause analysis, and operational improvement activities. • Contribute to improvements in monitoring, observability, automation, and operational processes. • Maintain operational documentation, runbooks, and knowledge articles.

Europe
$60K - $67K / year