Simplify Your Life. Supercharge Your Shop.
Senior Software Engineer, Python
Location
United States
Posted
82 days ago
Salary
0
Seniority
Senior
Job Description
Senior Software Engineer, Python
Tekmetric
• We are seeking a Senior Software Engineer with expertise in web scraping, data processing, and search technologies to help build a large-scale data ingestion and classification system. • You will be responsible for extracting data from diverse sources (web pages, APIs, PDFs), cleaning and normalizing it, and building search capabilities using ElasticSearch/OpenSearch. • You will work with Python, Scrapy, Airflow, Kubernetes, AWS, and Spark to create scalable, high-performance data pipelines. • Build and design large scale, distributed crawling bots (perhaps AI agents) and infrastructure that operate in an adversarial environment aiming at low operational overhead. • Develop and maintain data pipelines to extract data from large volumes of web pages, documents, PDFs (OCR), and APIs. • Help unify heterogeneous documents into a coherent data schema across varied source formats. • Preprocess and normalize raw data for downstream classification, ML/NLP, and search indexing. • Build APIs to expose structured, classified data via ElasticSearch/OpenSearch. • Collaborate with ML/NLP teams to integrate classification models into the pipeline. • Automate workflows using Apache Airflow and deploy solutions in Kubernetes on AWS. • Optimize and scale data pipelines using Spark (EMR) for processing large datasets.
Job Requirements
- 5+ years of experience in Python with building crawling/scraping solutions at scale.
- Experience working with APIs (REST), PDF processing (OCR, Tesseract, PyMuPDF etc.).
- Proficiency in data processing & search technologies (ElasticSearch/OpenSearch, NoSQL/SQL databases).
- Hands-on experience with Airflow and Spark (EMR) or similar distributed systems.
- Strong problem-solving skills in handling anti-scraping mechanisms and data scaling challenges.
- Hands-on experience with AWS or GCP.
Benefits
- Enjoy the flexibility of remote work
- Competitive base salaries that reflect your value.
- Generous Paid Time Off, because we know you do your best work when you're well-rested.
- Support for every stage of life—with paid maternity, parental bonding, and medical leave for you or your loved ones.
- Comprehensive health benefits, including Medical, Dental, Vision, and Prescription coverage. For employee only, we offer plans that cover 100% of premiums and we cover 50% of costs for families.
- Prioritizing your mental health: get free, confidential counseling through our partnership with BetterHelp.
- 401(k) Retirement Savings Plan with 100% employer match on contributions up to 6% - so your future self will thank you.
- Flexible Spending Accounts (FSA) and Health Savings Accounts (HSA) to make your money go further.
- Life and Accidental Death & Dismemberment (AD&D) Insurance for added peace of mind.
- Wellness on your terms: get up to $60/month toward fitness, mental health, or almost anything that helps you feel your best.
- After one year of employment, enjoy a $300 home office setup bonus to help make your space work for you.
- Keep growing with support for continuing education - we’re invested in your development.
Related Guides
Related Job Pages
More Backend Engineer Jobs
Database Engineer
UbimindsWe deliver custom-curated digital product teams with Staff Augmentation & Employer-of-Record Services in Brazil.
• Design and implement new features in a React/TypeScript codebase; • Manage routes and server-side rendering with React Router; • Build responsive, accessible UIs using Tailwind CSS and modular CSS; • Integrate with backend services via Axios (or similar) and maintain a clean API client layer; • Define and maintain TypeScript interfaces/models for data and service contracts; • Enforce code quality with ESLint, Prettier, lint-staged and Husky pre-commit hooks; • Write unit/integration tests (Jest, React Testing Library or equivalent); • Collaborate with product and design teams to translate requirements into polished UI; • Troubleshoot bugs, optimize performance, and participate in code reviews
• Develop and maintain applications using Java and React. • Implement microservices with a focus on scalability, performance, and security. • Integrate systems with cloud services, ensuring efficiency, reliability, and security. • Collaborate with cross-functional teams to define and deliver new features. • Participate in code reviews and ensure software quality. • Identify and resolve performance and scalability issues. • Contribute to architectures that support coexistence with legacy systems. • Work on event-driven solutions using tools such as Kafka. • Document processes, technical solutions, and architectural decisions. • Propose continuous improvements to processes and solutions with the team and the client.
• Atuar com o time de desenvolvimento na garantia de uma entrega de qualidade para nosso cliente; • Atuar com resolução de problemas de forma mais assertiva, com visão macro do negócio; • Ampliar os conhecimentos do time com elaboração de treinamentos/dojos
• Design, develop, unit test, and integrate fullstack application components. • Build scalable web apps using React on the frontend and Kotlin/Spring Boot on the backend. • Implement RESTful APIs, microservices, and integrations with enterprise systems. • Collaborate with UX/UI, product, architecture, and QA teams in an Agile environment. • Create and maintain architecture, design, and system documentation.
