MLabs LTD

Founded in 2018, MLabs is a private software engineering consultancy specializing in Haskell and Rust development with a focus on blockchain, artificial intelli

Research Crawling Engineer

Location

New York

Posted

41 days ago

Salary

$80K - $175K / year

Seniority

Senior

Job Description

Research Crawling Engineer

MLabs LTD

• Construct and maintain large-scale web crawlers across diverse domains. • Design high-throughput, fault-tolerant systems for data collection, managing volumes ranging from millions to billions of URLs per day. • Navigate anti-bot systems, rate limits, and dynamic, JavaScript-heavy websites. • Develop robust pipelines for data cleaning, deduplication, filtering, and normalization. • Build and maintain datasets specifically structured for research and machine learning model training. • Monitor and optimize crawl performance, coverage, and data quality through rapid iteration. • Collaborate with research teams to ensure data collection efforts align with modeling requirements. • Optimize infrastructure to ensure cost-efficiency, low latency, and reliability.

Job Requirements

  • Extensive programming experience in one or more of the following: Go, Rust, Python, Java, or C++.
  • Proven experience in building web crawlers or large-scale data pipelines.
  • Solid understanding of HTTP, networking protocols, and browser behavior.
  • Familiarity with distributed systems and parallel processing techniques.
  • Experience handling large datasets, ideally at the terabyte to petabyte scale.
  • Demonstrated ability to debug and maintain systems within unstable or adversarial environments.
  • Preferred Qualifications:
  • Experience with NLP pipelines or dataset curation for machine learning.
  • Familiarity with LLM pre-training data or retrieval systems.
  • Practical experience with headless browsers (e.g., Playwright, Puppeteer, or Chrome DevTools Protocol).
  • Knowledge of proxy systems, IP rotation, and large-scale request orchestration.
  • Background in data quality evaluation or benchmarking.
  • Experience running workloads on cloud or bare-metal infrastructure.

Benefits

  • Impactful Opportunity: Contribute to the development of a web-scale crawler and knowledge graph at the forefront of AI data accessibility.
  • High-Performance Culture: Join a lean, low-ego team that prioritizes high output and professional growth.
  • Remote Work: This position is part of a fully remote team, offering flexibility and autonomy.
  • Competitive Compensation: A package including a competitive salary, comprehensive benefits, and equity, commensurate with experience and the ability to operate at scale.

Related Categories

Related Job Pages

More Engineer Jobs

Syner-G logo

Sr. Automation Engineer

Syner-G

A career here is life-enhancing. At Syner-G, we enable our people to build careers that impact positively on their quality of life. Through our expertise, insight, consulting and management skills, we accelerate breakthrough science and delivery of life-enhancing therapies to more patients. We work across a diverse range of clients and projects, supporting many organizations from the most critical phases of the drug discovery and approval process through to commercialization. Syner-G was recently honored with BioSpace's prestigious "Best Places to Work" 2026 award, for the third consecutive year, along with many other award-winning programs to make a career here truly life-enhancing. These recognitions are a testament to our commitment to fostering a positive and engaging work environment for our employees, with a particular emphasis on culture, career growth and development opportunities, financial rewards, leadership and innovation. At Syner-G, we recognize that our team members are our most valuable asset. Join us in shaping the future, where your talents are valued, and your contributions make a meaningful impact.

Engineer41 days ago
Full TimeRemoteTeam 201-500

Role Description Syner-G is seeking a Sr. Automation Engineer with 7–11 years of experience and strong hands-on expertise in Ignition SCADA, PLC-level programming (Rockwell and/or Siemens), and exposure to modern digital or MES-lite system architectures. This role leads the oversight, documentation, and lifecycle management of automation and digital systems in GMP-regulated environments. The Sr. Automation Engineer will act as a senior technical authority, driving automation strategy, ensuring system reliability and compliance, and supporting complex troubleshooting and integration efforts across client engagements. Work location may require travel to client sites up to 100%, depending on project needs and client expectations. Key Responsibilities - Lead development, maintenance, and review of automation system documentation, including SOPs, design documents, and lifecycle deliverables such as DS, FS, CA, RTM, and RA. - Oversee system lifecycle activities, periodic reviews, and audit readiness aligned with regulatory expectations and industry best practices. - Author, review, and execute complex validation documentation including IOQs, engineering tests, protocols, and summary reports. - Drive CSV and SDLC processes with strong traceability, risk assessments, and compliance rigor. - Perform advanced data and system reviews, including alarm rationalization, user access audits, and performance evaluations. - Own and manage automation-related change records, including impact assessments, stakeholder coordination, and documentation updates. - Oversee and coordinate maintenance activities, work orders, and system enhancements across automation and digital platforms. - Collaborate with internal teams and external vendors to resolve complex issues, support upgrades, and implement system improvements. - Serve as a senior liaison between Facilities, Quality, IT, Operations, and external partners. - Mentor junior and mid-level automation staff, providing technical leadership and oversight. - Lead audit preparation activities and ensure systems remain inspection-ready at all times. - Provide subject-matter leadership in Ignition SCADA configuration, scripting, module management, and integration with PLCs and MES-lite systems. - Support PLC and SCADA programming, troubleshooting, and optimization across Rockwell, Siemens, and related control platforms. - Contribute to digital transformation initiatives, including MES-lite deployments, data modeling, historian integration, and OT/IT convergence efforts. Qualifications - BS or MS in Engineering, Life Sciences, or a related technical field preferred. - 7–11 years supporting and leading automation initiatives in GMP-regulated biotech or pharmaceutical environments. - Hands-on experience with Ignition (Inductive Automation) for SCADA/HMI development, scripting, gateway configuration, and system integration. - PLC-level programming experience with Rockwell ControlLogix or CompactLogix and/or Siemens S7 or TIA Portal. - Experience with BAS/BMS, EMS, or process automation platforms such as Siemens Desigo, DeltaV, Rockwell, or PLC/SCADA systems. - Exposure to modern digital or MES-lite stacks such as Ignition Perspective, Sepasoft modules, lightweight MES workflows, data lakes, or historians. - Extensive background in GMP documentation, validation, and CSV/SDLC processes. - Demonstrated ability to lead technical efforts, manage complex automation projects, and support SMEs through strong documentation, coordination, and compliance execution. Requirements - Strong understanding of automation system architecture, control logic, alarm logic, and point configuration review. - Strong Ignition development skills including Vision or Perspective, scripting, UDTs, tag structures, and gateway configuration. - Advanced skill in writing, reviewing, and managing controlled documents and validation protocols. - Strong leadership, communication, organization, and cross-functional coordination abilities. - Ability to manage multiple high-priority tasks in a fast-paced, compliance-driven environment. Benefits - Market competitive base salary and annual incentive plan. - Robust benefit offerings. - Ongoing recognition and career development opportunities. - Generous flexible paid time off program. - Company-paid holidays. - Flexible working hours and fully remote work options for most positions. - Office locations in Greater Boston; San Diego, CA; Boulder, CO; and India for those preferring a physical work location. Legal Statement Syner-G is proud to be an Equal Employment Opportunity and Affirmative Action employer. All employment decisions, including the recruiting, hiring, placement, training availability, promotion, compensation, evaluation, disciplinary actions, and termination of employment (if necessary) are made without regard to the employee’s race, color, creed, religion, sex, pregnancy or childbirth, personal appearance, family responsibilities, sexual orientation or preference, gender identity, political affiliation, source of income, place of residence, national or ethnic origin, ancestry, age, marital status, military veteran status, unfavorable discharge from military service, physical or mental disability, or on any other basis prohibited by applicable law. Syner-G is an E-Verify employer.

Worldwide
Job Closed
Intertek logo

Staff Engineer - Construction

Intertek

Bringing Quality, Safety and Sustainability to life

Engineer41 days ago
Full TimeRemoteTeam 10,001+Since 1885H1B Sponsor

Role Description Intertek PSI is planning to add a Staff Engineer to our Building and Construction team in Portland, OR. The target candidate population for this role is a newly graduated engineering professional looking to gain experience in the Construction Industry. The Staff Engineer role performs various evaluations, documents observations related to sample collections and testing, and may represent the company in client interactions. The ideal candidate for this role will be comfortable working independently, eager to gain new experiences, and enjoy working in a fast-paced environment. Shift/Schedule: Monday - Friday (8am-5pm) What you’ll do: - Performs a variety of assignments that include independent evaluations using standard techniques, procedures and criteria using judgment to make minor adoptions and modifications to these standards. - Performs preliminary report writing and review. - Reviews project plans and specifications prepared by others. - Attends client site meetings. - Communicates effectively with client and project teams. - Works on one or multiple projects at a time. - Trains Technicians. - May perform on-site observations, sample collection, and specific tests. - May work both in the field and laboratory regularly. - Must be aware of, and adhere to, safety practices and policies to ensure your own safety, as well as the safety of others who may be affected by your actions at work. Qualifications - Bachelors Science Degree from accredited engineering /ABET school. - No experience required. - Ability to communicate and interact effectively in verbal & written communication. - Ability to work off shifts and overtime. - May travel up to 25% of time. - Valid driver’s license and reliable driving record is required. - EIT certification Preferred. Requirements - Ability to lift, move, push and pull 30 to 50 pounds frequently. Occasionally, over 50 pounds with assistance. - Ability to receive detailed information through oral communication (hearing) and to make the discriminations in sound. - Ability to kneel and squat occasionally. - Ability to walk and stand for long periods of time. - Ability to work outdoors in adverse weather conditions (hot and cold). - Ability to climb occasionally. Benefits - Competitive compensation packages. - Medical, dental, vision, life, disability insurance. - 401(k) with company match. - Generous vacation / sick time (PTO). - Tuition reimbursement. - More benefits available.

United States
Job Closed

Customer Experience Engineer (APIs) - EST

HiBob

HiBob is a modern HR technology company focused on transforming the way organizations operate in today’s dynamic workplace. Its platform streamlines core HR processes, enhances e

Engineer41 days ago
Full TimeRemoteTeam 1,350Since 2015

Job Description We're hiring a Customer Experience Engineer (Technical) to act as the highest level of technical escalation within our Customer Experience organization. This is a deeply technical role focused on debugging complex system issues , investigating product behavior , and partnering closely with Engineering to drive reso lution and long-term improvements. You won't just solve tickets-you'll analyze systems, identify root causes, and improve how we operate at scale . The base salary range for this role is $85,000 - $108,000. There is a strong preference to fill this role in NYC with a hybrid schedule out of our office, but we will consider exceptional candidates who are based in the EST time zone and open to working remotely. Location Eligibility: While can be a remote position, HiBob is currently authorized to hire in the following states: CA. CO, CT, DC, FL, GA, IL, IN, KS, MA, MD, MN, NC, NH, NJ, NV, NY, OH, OK, OR, PA, RI, SC, TN, TX, UT, VA, WA. Job Requirements - 4-6+ years in Technical Support Engineering, Support Engineering, or similar roles - Strong SQL skills (joins, data validation, debugging datasets) - Deep understanding of APIs, webhooks, and integrations - Experience with logs and monitoring tools (e.g., Datadog, Splunk, etc.) - Proven experience handling high-severity technical incidents - Strong problem-solving mindset focused on root cause analysis Nice To Have: - Experience in SaaS or microservices environments - Background in software development - Familiarity with observability and incident management practices - Experience building automation or internal tools Job Responsibilities Own complex technical escalations - Lead investigations across APIs, integrations, webhooks, and backend systems - Perform root cause analysis using logs, SQL, and internal tooling - Reproduce issues and isolate failures across distributed environments - Partner with Engineering on bug triage and resolution Debug at the system level - Analyze logs, traces, and database behavior to diagnose issues - Investigate API failures, authentication issues, and data inconsistencies - Work with observability tools to track incidents across services Drive reliability and improvement - Identify recurring patterns and propose systemic fixes - Build debugging frameworks and escalation playbooks - Improve issue reproducibility and reduce time to resolution - Contribute to automation and internal tooling Enable the broader team - Mentor support engineers on advanced troubleshooting techniques - Create technical documentation and internal knowledge resources - Act as a bridge between Customer Experience and Engineering Benefits HiBob is a village filled with amazing people, and we're especially proud of that. It's a place where Bobbers can be themselves. We're about fun, dreams, hopes, and ambition, just as much as we are about precision, growth, and top performance. Becoming a Bobber means you'll receive competitive compensation, benefits, and pre-IPO equity alongside all of this: - Stock options at a high-growth unicorn startup - 100% subsidized medical, dental, and vision coverage for employees - 401(k) with a 3% company match starting from Day 1 - Hybrid working model for those who are in the NYC metro area. - Work from home allowance to get your home office set up! - Temporary remote work-from-anywhere in the world for up to 2 months after 6 months of employment - Annual Headspace subscription and wellness benefits - Two social impact days per year for volunteering - Bob balance days - 4 additional days within a calendar year - Enjoy a company-wide long weekend at the beginning of each quarter - Employee referral program - $2,500 bonus for each successful referral, with an additional ambassador bonus - Fun and frequent social events (in-person and virtual) - We love birthdays - take the day off and receive a special gift - Dog-friendly office If this sounds like something you've been looking for, we'd love to have you. Come on, join our village!

United States
$85K - $108K / year
Job Closed
EQT Corporation logo

SCADA Engineer I

EQT Corporation

Our mission is to deliver cheaper, more reliable, cleaner energy to the world.

Engineer41 days ago
Full TimeRemoteTeam 501-1,000Since 1888H1B No Sponsor

• Support daily SCADA operations, including system maintenance, troubleshooting, user administration, and application development • Gather and interpret business requirements and translate them into technical designs, enhancements, and integrations • Design, develop, and modify SCADA displays and system configurations for new and existing facilities, including: • Device/point and polling configuration • Display creation and data validation • Advanced control interfaces • Management of associated Salesforce cases • Provide advanced support for MQTT, Edge devices, and RTU/PLC communications ensuring reliable data flow across systems • Ensure all SCADA displays, alarms, and communications comply with EQT’s internal SOPs and Production standards • Leverage a deep understanding of system architecture to solve complex technical problems and improve system performance and usability • Perform system audits and analyses, including: • Critical point validation • SOP compliance checks • Server data and event log reviews • Develop and deliver training materials and sessions for internal users and SCADA Specialists • Lead/Coordinate M&A activity, system upgrades, maintenance, and performance improvements, ensuring minimal operational disruption through testing and validation • Collaborate with vendors and internal IT teams to support infrastructure and application initiatives • Provide input into SCADA architecture, cybersecurity, and high-availability (redundancy) design, and assist in system buildout and testing • Contribute to SCADA design strategy across applications and infrastructure

California + 8 moreAll locations: California | Connecticut | Illinois | Louisiana | New Jersey | New York | Massachusetts | Michigan | Tennessee