Take Control of Your Business and Execute Your Vision with Ease - Hire Affordable and Qualified Nearshore Staff
Data Engineer – Web Scraping, LLM Pipelines, Scalable Data Infrastructure
Location
Argentina
Posted
8 days ago
Salary
0
Seniority
Senior
Job Description
Data Engineer – Web Scraping, LLM Pipelines, Scalable Data Infrastructure
NIR-YU
• Build new structured datasets, including scraping accelerators, Form D filings and dynamic web sources. • Develop automated ETL pipelines that parse, clean and transform content using LLMs. • Define and maintain database schemas in Supabase or PostgreSQL. • Create evaluation frameworks to measure and compare LLM performance across pipeline components. • Contribute to the design of scalable data architectures using GCP services. • Improve reliability, observability and deployment workflows for scraping and data processing systems.
Job Requirements
- 4+ years of experience building data pipelines, backend services and automated data processing systems.
- Strong background in web scraping with tools like Scrapy, Playwright or similar.
- Experience deploying pipelines on cloud platforms such as GCP or AWS.
- Solid knowledge of ETL frameworks, workflow orchestration (Airflow) and modern data stores (BigQuery, PostgreSQL).
- Comfortable working with Docker and API frameworks like FastAPI.
- Clear, fluent communication in English.
Related Guides
Related Categories
Related Job Pages
More Infrastructure Engineer Jobs
AI Infrastructure & Platform Operations Engineer
MirantisStrategic open source infrastructure for containers and virtual machines.
• Monitor, operate, and support production AI infrastructure platforms. • Investigate and resolve infrastructure, networking, hardware, and platform-related incidents. • Support NVIDIA GPU infrastructure and associated platform services. • Monitor and troubleshoot Kubernetes-based environments. • Investigate performance, availability, and reliability issues across infrastructure and platform components. • Collaborate with engineering teams, hardware vendors, datacenter personnel, and service delivery teams to resolve technical issues. • Participate in incident response, root cause analysis, and operational improvement activities. • Contribute to improvements in monitoring, observability, automation, and operational processes. • Maintain operational documentation, runbooks, and knowledge articles.
Senior AI Infrastructure & Platform Operations Engineer
MirantisStrategic open source infrastructure for containers and virtual machines.
• Lead the investigation and resolution of complex infrastructure, networking, and platform-related incidents. • Act as a senior escalation point for operational teams during critical service-impacting events. • Support large-scale NVIDIA GPU infrastructure and high-performance networking environments. • Troubleshoot complex Linux, Kubernetes, networking, storage, and hardware-related issues. • Analyze platform performance, capacity, stability, and reliability trends to proactively identify risks. • Lead root cause analysis activities and drive long-term corrective actions. • Collaborate with engineering teams, hardware vendors, and datacenter personnel to resolve complex technical challenges. • Participate in major incident management and service restoration activities. • Provide technical leadership for Kubernetes platform operations and supporting infrastructure services. • Drive improvements in platform reliability, observability, monitoring, and operational processes. • Identify opportunities to automate repetitive operational activities and improve operational efficiency. • Contribute to operational readiness reviews, infrastructure changes, upgrades, and service introductions. • Support the adoption and operation of AI-powered infrastructure services and operational capabilities through k0rdent AI. • Evaluate emerging technologies and operational practices to improve service delivery and platform resilience. • Mentor and support AI Infrastructure & Platform Operations Engineers. • Share technical knowledge through documentation, training sessions, and operational reviews. • Develop and maintain operational standards, runbooks, troubleshooting guides, and best practices. • Help define operational processes, escalation paths, and service reliability standards. • Act as a trusted technical advisor during operational planning and service improvement initiatives.
AI Infrastructure & Platform Operations Engineer
MirantisStrategic open source infrastructure for containers and virtual machines.
• Monitor, operate, and support production AI infrastructure platforms. • Investigate and resolve infrastructure, networking, hardware, and platform-related incidents. • Support NVIDIA GPU infrastructure and associated platform services. • Monitor and troubleshoot Kubernetes-based environments. • Investigate performance, availability, and reliability issues across infrastructure and platform components. • Collaborate with engineering teams, hardware vendors, datacenter personnel, and service delivery teams to resolve technical issues. • Participate in incident response, root cause analysis, and operational improvement activities. • Contribute to improvements in monitoring, observability, automation, and operational processes. • Maintain operational documentation, runbooks, and knowledge articles.
AI Infrastructure, Platform Operations Engineer
MirantisStrategic open source infrastructure for containers and virtual machines.
• Monitor, operate, and support production AI infrastructure platforms. • Investigate and resolve infrastructure, networking, hardware, and platform-related incidents. • Support NVIDIA GPU infrastructure and associated platform services. • Monitor and troubleshoot Kubernetes-based environments. • Investigate performance, availability, and reliability issues across infrastructure and platform components. • Collaborate with engineering teams, hardware vendors, datacenter personnel, and service delivery teams to resolve technical issues. • Participate in incident response, root cause analysis, and operational improvement activities. • Contribute to improvements in monitoring, observability, automation, and operational processes. • Maintain operational documentation, runbooks, and knowledge articles.

