Job Closed

This listing is no longer active.

Mindrift logo
Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Freelance Web Scraping Engineer (Vibe Coding)

Location

New York

Posted

80 days ago

Salary

0

Seniority

Mid Level

EnglishJavaScript

Job Description

Freelance Web Scraping Engineer (Vibe Coding)

Mindrift

This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English. Mindrift is looking for highly skilled Vibecode specialists to join the Tendem project (https://tendem.ai/) and drive specialized data scraping workflows within our hybrid AI + human system. In this role, as an AI Pilot – that’s how we refer to this role at Mindrift – you’ll collaborate with Tendem Agents that handle repetitive tasks, while you provide critical thinking, domain expertise, and quality control to deliver accurate and actionable results. This part-time remote opportunity is ideal for technical professionals with hands-on experience in web scraping, data extraction and processing. What we do The Mindrift platform connects specialists with AI projects from major tech innovators. Our mission is to unlock the potential of Generative AI by tapping into real-world expertise from across the globe. About the Role This is a freelance role for a Tendem project. As a Vibe Code specialist, you'll handle data scraping tasks requiring technical precision for web extraction and processing, utilizing various tools such as our provided Apify and OpenRouter alongside your own resourceful approaches. Key Responsibilities - Own end-to-end data extraction workflows across complex websites, ensuring complete coverage, accuracy, and reliable delivery of structured datasets. - Leverage internal tools (Apify, OpenRouter) alongside custom workflows to accelerate data collection, validation, and task execution while meeting defined requirements. - Ensure reliable extraction from dynamic and interactive web sources, adapting approaches as needed to handle JavaScript-rendered content and changing site behavior. - Enforce data quality standards through validation checks, cross-source consistency controls, adherence to formatting specifications, and systematic verification prior to delivery. - Scale scraping operations for large datasets using efficient batching or parallelization, monitor failures, and maintain stability against minor site structure changes. Compensation On this project, contributors can earn up to $32 per hour equivalent, depending on their level and pace of contribution. Compensation varies across projects depending on scope, complexity, and required expertise. Please note that other projects on the platform may offer different earning levels based on their requirements. How to get started Simply apply to this post, qualify, and get the chance to contribute to projects that match your technical skills, on your own schedule. From coding and automation to fine-tuning AI outputs, you’ll play a key role in advancing AI capabilities and real-world applications.

Job Requirements

  • At least 1 year of relevant experience in data analysis, AI automation, data engineering, or software development is desirable.
  • Bachelor's or Master’s Degree in Engineering, Applied Mathematics, Computer Science, or related technical fields is a plus.
  • Python web scraping: Build reliable scraping scripts using BeautifulSoup, Selenium (or equivalents) for multi-level sites, dynamic JS content (infinite scroll, AJAX), and API endpoints via provided proxy.
  • Data extraction expertise: Navigate complex hierarchical structures (regions → companies → details), handling archived pages and varied HTML formats.
  • Data processing: Clean, normalize, and validate scraped data; deliver high-quality datasets in well-structured formats (CSV, JSON, Google Sheets) with clear, consistent presentation.
  • Hands-on experience with LLMs and AI frameworks to enhance automation and problem-solving.
  • Strong attention to detail and commitment to data accuracy.
  • Self-directed work ethic with ability to troubleshoot independently.
  • English proficiency: Upper-intermediate (B2) or above (required).

Benefits

  • Why this freelance opportunity might be a great fit for you?
  • Work fully remote on your own schedule with just a laptop and stable internet connection.
  • Gain hands-on experience in a unique hybrid environment where human expertise and AI agents collaborate seamlessly — a distinctive skill set in a rapidly growing field.
  • Participate in performance-based bonus programs that reward high-quality work and consistent delivery.

Related Categories

Related Job Pages

More Data Engineer Jobs

Fidelity Investments logo

Principal Data Engineer

Fidelity Investments

Here to help you grow your savings and invest in the life you want.

Data Engineer80 days ago
Full TimeRemoteTeam 10,001+Since 1946H1B Sponsor

Job Description: Position Description: ***Applicants are permitted to work remotely from an at home worksite anywhere in the United States.*** Designs and implements single and multi-tenant data solutions using cloud database technologies. Develops and implements Proof of Concepts (PoCs), frameworks, and real-time pipelines, using SQL, Amazon Web Services (AWS), Snowflake, and Python. Manages data integration, ensuring high data quality and accessibility, using Snowflake and Extract, Transform, Load (ETL) / Extract, Load, Transform (ELT) tools (Informatica). Builds RESTful APIs using Spring Boot, Java, and Python. Develops scalable and robust data solutions for websites using SQL, MySQL, and SQLMI. Builds, maintains, and performs tunning of stored procedures. Assists in planning and execution of user acceptance testing. Proposes modifications and improvements based on user needs. Primary Responsibilities: - Participates in architecture design teams. - Defines and implements application level architecture. - Develops applications on complex projects, components, and subsystems for the division. - Recommends development testing tools and methodologies and reviews and validates test plans. - Responsible for QA readiness of software deliverables. - Develops comprehensive documentation for multiple applications or subsystems. - Establishes full project life cycle plans for complex projects across multiple platforms. - Responsible for meeting project goals on-time and on-budget. - Advises on risk assessment and risk management strategies for projects. - Plans and coordinates project schedules and assignments for multiple projects. - Acts as a primary liaison for business units to resolve various project/technology issues. - Provides technology solutions to daily issues and technical evaluation estimates on technology initiatives. - Advises senior management on technical strategy. - Mentors junior team members. - Develops comprehensive documentation for multiple applications in support of corporate initiatives. - Performs independent and complex technical and functional analysis for multiple simultaneous projects. - Confers with systems analysts and other software engineers/developers to design systems and to obtain information on project limitations and capabilities, performance requirements and interfaces. Education and Experience: Bachelor’s degree in Computer Science, Engineering, Information Technology, Information Systems, or a closely related field (or foreign education equivalent) and five (5) years of experience as a Principal Data Engineer (or closely related occupation) developing ETL framework to transfer files, creating PL/SQL scripts, and creating implementation plan during production implementation. Or, alternatively, Master’s degree in Computer Science, Engineering, Information Technology, Information Systems, or a closely related field (or foreign education equivalent) and three (3) years of experience as a Principal Data Engineer (or closely related occupation) developing ETL framework to transfer files, creating PL/SQL scripts, and creating implementation plan during production implementation. Skills and Knowledge: Candidate must also possess: - Demonstrated Expertise (“DE”) performing conceptual data modeling, designing, and developing multi-tenant data solutions for websites using MySQL Workbench, and DataGrip; coordinating database operations and data storage and retrieval using SQL, stored procedures, and triggers on MySQL, Snowflake, and Cloud technologies. - DE performing ETL, ELT, and data integration processes with complex data transformations, using SQL, Python, PowerDesigner, Unix shell scripting, and JSON/ XML Parsing; performing full life cycle analysis and development of Operational and Transactional applications built using Angular; and tuning SQL query performance using TOAD, SQL Developer, and MySQL workbench. - DE interpreting requirements within an Agile environment and developing APIs to move data to and from multiple sources including files and Application Programming Interfaces (APIs) to on-premise or Cloud databases (AWS or Azure) using SpringBoot, Java, and Python. - DE applying Continuous Integration/Continuous Deployment (CI/CD) practices to database releases -- automated deployments -- using Jenkins, JenkinsCore and Urban Code Deploy (uDeploy), applying SDLC best practices -- source code management (Git, Stash, and SourceTree) and documentation using software -- Atlassian Confluence and draw.io. Salary: $141,450.00 - $166,000.00/year. #PE1M2 #LI-DNI Certifications: Category: Information Technology Most roles at Fidelity are Hybrid, requiring associates to work onsite every other week (all business days, M-F) in a Fidelity office. This does not apply to Remote or fully Onsite roles. Some roles may have unique onsite requirements. Please consult with your recruiter for the specific expectations for this position. Please be advised that Fidelity’s business is governed by the provisions of the Securities Exchange Act of 1934, the Investment Advisers Act of 1940, the Investment Company Act of 1940, ERISA, numerous state laws governing securities, investment and retirement-related financial activities and the rules and regulations of numerous self-regulatory organizations, including FINRA, among others. Those laws and regulations may restrict Fidelity from hiring and/or associating with individuals with certain Criminal Histories.

United States
$141K - $166K / year
Leidos logo

Data Transport NiFi Lead

Leidos

Leidos is an innovation company rapidly addressing the world’s most vexing challenges in national security and health.

Data Engineer80 days ago
Full TimeRemoteTeam 10,001+Since 1969H1B Sponsor

Leidos was awarded the U.S. Air Force Cloud One Architecture and Common Shared Services contract. Leidos is seeking a Data Transport NiFi Lead. This is an exciting opportunity to use your experience to modernize a leading, global-scale multi-cloud environment in support of a critical mission, supporting USAF system resiliency, security, and cost effectiveness. Location: These positions will be remote. Candidates will be required to work onsite as needed. Preferred candidates will be located near Hanscom AFB (Boston, MA) or work in Huntsville, AL. Primary Responsibilities Could Include: - Lead systems requirements analysis, functional analysis, and requirements allocation for complex cloud and hybrid environments. - Design and implement scalable, resilient, and secure cloud architectures in AWS, Azure, Google Cloud Platform, and/or Oracle Cloud Infrastructure. - Conduct technical trade studies and evaluate alternatives considering cost, risk, performance, security, and lifecycle supportability. - Lead the development and maintenance of Infrastructure as Code (IaC) and automation frameworks using tools such as Apache NiFi, Terraform, Ansible, or CloudFormation. - Architect, deploy, and integrate cloud-native applications and supporting services. - Design and support containerized platforms and orchestration solutions using Kubernetes, Docker, and related technologies. - Optimize system availability, performance, scalability, and reliability across distributed cloud environments. - Implement and oversee security best practices, including identity and access management, logging, monitoring, vulnerability management, and compliance controls. - Support system integration, verification, validation, testing, and quality assurance activities across multiple technical domains. - Lead cloud modernization initiatives, including migration of legacy systems and adoption of cloud-native architectures. - Troubleshoot complex system issues, perform root cause analysis, and drive resolution across teams. - Develop and maintain technical documentation, architecture artifacts, and operational procedures. - Collaborate with internal engineering teams, program leadership, customers, and external partners or vendors to deliver integrated solutions. - Provide technical leadership, coaching, and design review support to junior and mid-level engineers. - Influence engineering standards, DevOps/DevSecOps practices, and technical processes. Minimum Qualifications: - Bachelor’s degree with 8+ years of relevant experience, or Master’s degree with 6+ years of relevant experience. A Doctorate in a related technical domain may be considered. - Interim Secret clearance required to start; ability to obtain and maintain a Secret clearance. - U.S. citizenship required. - CompTIA Security+ or equivalent certification (IAT Level II). - Expert-level experience designing and supporting cloud environments (AWS, Azure, GCP, or OCI). - Strong experience with Infrastructure as Code, automation, and DevOps/DevSecOps practices. - Hands-on experience with CI/CD pipelines and cloud-native application architectures. - Advanced knowledge of Linux/Unix systems, networking, and cloud security concepts. - Proficiency in one or more scripting or programming languages (e.g., Python, Bash, Go). - Excellent written and verbal communication skills. Preferred Qualifications: - Experience supporting DoD or USAF cloud environments (e.g., Cloud One, Platform One). - Experience designing and implementing Zero Trust cloud architectures. - Experience with monitoring and logging platforms such as Elastic, Splunk, Prometheus, or Grafana. - Strong understanding of identity and access management solutions (Active Directory, Azure AD, IAM). - Experience with container orchestration platforms such as Kubernetes and OpenShift. - Cloud certifications in AWS, Azure, Google, or Oracle. - Experience collaborating with vendors and external partners on cloud solutions. - Certification Examples - AWS Certified Solutions Architect (Professional), Azure Solutions Architect (Expert), MCSE (Server), Certified AWS SysAdmin, AWS Certified Cloud Practitioner, AWS Certified Developer, AWS Certified Solutions Architect (Dev/Associate), AWS Certified DevOps Engineer, AWS Certified Advanced Networking, AWS Certified Security, Azure Developer Associate, Azure Solutions Architect. C1NACSS If you're looking for comfort, keep scrolling. At Leidos, we outthink, outbuild, and outpace the status quo — because the mission demands it. We're not hiring followers. We're recruiting the ones who disrupt, provoke, and refuse to fail. Step 10 is ancient history. We're already at step 30 — and moving faster than anyone else dares. Original Posting: March 25, 2026 For U.S. Positions: While subject to change based on business needs, Leidos reasonably anticipates that this job requisition will remain open for at least 3 days with an anticipated close date of no earlier than 3 days after the original posting date as listed above. Pay Range: Pay Range $107,900.00 - $195,050.00 The Leidos pay range for this job level is a general guideline only and not a guarantee of compensation or salary. Additional factors considered in extending an offer include (but are not limited to) responsibilities of the job, education, experience, knowledge, skills, and abilities, as well as internal equity, alignment with market data, applicable bargaining agreement (if any), or other law.

United States
$107K - $195K / year
Job Closed
Logicalis Spain logo

Junior Data Engineer – DataStage

Logicalis Spain

Somos Arquitectos Del Cambio, ayudamos a las organizaciones a tener éxito en un mundo cada vez más digitalizado.

Data Engineer80 days ago
Full TimeRemoteTeam 1,001-5,000H1B No Sponsor

• Participar en el diseño, desarrollo y mantenimiento de procesos ETL utilizando IBM DataStage. • Apoyar la integración de datos desde múltiples fuentes (bases de datos, APIs, ficheros, etc.). • Trabajar con el equipo para optimizar flujos de datos y mejorar la eficiencia de las cargas. • Realizar pruebas, validaciones y documentación técnica. • Colaborar en la resolución de incidencias y soporte a procesos batch. • Contribuir a la implementación de buenas prácticas de gobierno y calidad del dato.

Spain
Job Closed
Keyrus logo

Data Engineer, AWS

Keyrus

#MakeDataMatter #HumanizingTheFuture

Data Engineer80 days ago
Full TimeRemoteTeam 1,001-5,000Since 1996H1B Sponsor

• Act as a bridge between business, technology, and data teams • Develop, maintain, and optimize data pipelines using AWS Glue • Participate in the migration and modernization of legacy processes

United States
Job Closed