Job Closed
This listing is no longer active.
MetLife is a leading insurance and financial services company based in New York, New York. The company and its affiliates specialize in employee benefits and li
Big Data Engineer II
Location
India
Posted
105 days ago
Salary
0
Seniority
Senior
Job Description
Big Data Engineer II
MetLife
Description and Requirements GG10.2 Data Engineer plays a critical role in data and analytics life cycle and significantly contributes to production grade data and analytics solutions. The role requires one to demonstrate Big Data, Engineering and Cloud expertise.It is an individual contributor role, expected to independently function 5-8+ years of relevant experience Bachelors degree in computer science, information technology or equivalent educational qualification • Design, build, and maintain robust ETL/ELT pipelines on cloud(Azure) or on-prem to collect, ingest and store large volumes of structured and unstructured data for batch/real time processing• Monitor, optimize, and troubleshoot data pipelines to ensure reliability, scalability, and performance• Ensure data processing, quality, security, and compliance guidelines, policies and standards are followed• Collaborate with multiple partners from Business, Technology, Operations and D&A capabilities (Data Governance, Data Quality, Data Modeling, Data Architecture, Data science, DevOps, BI & insights) • SQL, Python/Scala• NoSql and distributed databases (Hbase, Cosmos DB)• ETL pipleine development• Big Data Frameworks : Apache Spark, Hadoop, Hive• Cloud platforms: Azure data factory, Eventhub, Azure functions, Synapse, Databricks• Datawarehouses, data marts, data lakes• Medallion architecture• Performance tuning, optimization, and data quality validation• Real-time and batch data processing , streaming pieplines with Spark• Communication skills, analytical skills, structured problem-solving skills.,• Partner, Stakeholder engagement experience • DevOps practices: Git, AzureDevops, CI/CD pipelines• Unix shell scripting, MongoDB, Nifi• Exposure to Gen AI technology and tools About MetLife Recognized on Fortune magazine's list of the "World's Most Admired Companies" and Fortune World's 25 Best Workplaces™, MetLife, through its subsidiaries and affiliates, is one of the world's leading financial services companies; providing insurance, annuities, employee benefits and asset management to individual and institutional customers. With operations in more than 40 markets, we hold leading positions in the United States, Latin America, Asia, Europe, and the Middle East. Our purpose is simple - to help our colleagues, customers, communities, and the world at large create a more confident future. United by purpose and guided by our core values - Win Together, Do the Right Thing, Deliver Impact Over Activity, and Think Ahead - we're inspired to transform the next century in financial services. At MetLife, it's #AllTogetherPossible . Join us! #BI-Hybrid
Benefits
- 401(K), 401(K) matching, Adoption Assistance, Childcare benefits, Commuter benefits, Company equity, Company sponsored family events, Continuing education stipend, Dedicated diversity and inclusion staff, Dental insurance, Disability insurance, Documented equal pay policy, Volunteer in local community, Family medical leave, Flexible Spending Account (FSA), Flexible work schedule, Generous parental leave, Generous PTO, Company-sponsored happy hours, Health insurance, Highly diverse management team, Open door policy, Life insurance, Charitable contribution matching, Mentorship program, Paid volunteer time, Online course subscriptions available, Onsite gym, Open office floor plan, Paid holidays, Pair programming, Paid sick days, Onsite office parking, Partners with nonprofits, Performance bonus, Pet insurance, Promote from within, Recreational clubs, Lunch and learns, Remote work program, Team based strategic planning, OKR operational model, Team workouts, Continuing education available during work hours, Tuition reimbursement, Mandated unconscious bias training, Vision insurance, Wellness programs, Mental health benefits, Home-office stipend for remote employees, Diversity employee resource groups, Hiring practices that promote diversity, Employee resource groups, Employee-led culture committees, Hybrid work model, Employee awards, Floating holidays
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
When our values align, there's no limit to what we can achieve. At Parexel, we all share the same goal - to improve the world's health. From clinical trials to regulatory, consulting, and market access, every clinical development solution we provide is underpinned by something special - a deep conviction in what we do. Each of us, no matter what we do at Parexel, contributes to the development of a therapy that ultimately will benefit a patient. We take our work personally, we do it with empathy and we're committed to making a difference. This role is responsible for collaborating with the R&D function within Corp IT Data and Analytics team, including software and infrastructure design, ensuring quality and performance standards are met. Key Accountabilities: Data analysis: - Applies data analysis, data modelling, and quality assurance techniques, based upon a detailed understanding of business processes, to establish, modify or maintain data structures and associated components (entity descriptions, relationship descriptions, attribute definitions). Advises database designers and other application development team members on the details of data structures and associated components. Database design: - Develops and maintains specialist knowledge of database concepts, object and data modelling techniques and design principles and a detailed knowledge of database architectures, software and facilities. Analyses data requirements to establish, modify or maintain object/data models. Evaluates potential solutions, demonstrating, installing and commissioning selected products. Programming/ software development: - Designs, codes, tests, corrects and documents complex programs and scripts from agreed specifications, and subsequent iterations, using agreed standards and tools, to achieve a well-engineered result. Takes part in reviews of own work and leads reviews of colleagues' work. Systems design: - Specifies user/system interfaces and translates logical designs into physical designs taking account of target environment, performance security requirements and existing systems. Produces detailed designs and documents all work using required standards, methods and tools, including prototyping tools where appropriate. Key Responsibilities: - Design and develop ETL mappings and workflows using Qlik or Informatica. - Develop technical specifications for data loading rules. - Mentor and provide technical guidance for Software Engineers. - Design, implement, and architect software products. - Design, implement, and architect product extensions for customer specific requirements. - Adhere to and produce artifacts (technical specifications, architectural specifications, data models, installation and qualification guides, and release notes) for the software development process. - Support technical consulting activities to clients in the pharmaceutical industry. - Design and support clinical trials information sharing solutions. - Provide application support for existing products. - Help with release management, using versioning tools such as SVN, and executing live deployments. Skills: - Willingness to work in a matrix environment and to value the importance of teamwork. - Understanding of common system architecture frameworks and concepts. - Technical awareness of Security and Application Software Development. - Excellent interpersonal, verbal and written communication skills. - Client focused approach to work. Knowledge and Experience: - Proven relevant experience in systems integration and implementation, or software development, or a related discipline. - Demonstrated ability to liaise with multiple stakeholders. - Experience with Microsoft Azure components in particular, Azure SQL, Azure Analysis Services, Synapse. - Experience implementing ETL solutions, particularly utilizing Qlik Replicate and Compose. - Strong Database Development Skills, particularly implementing SQL solutions in Azure SQL, Synapse or Databricks. environments. Good experience with large datasets. - Experience improving Data Quality in Data Warehouses, particularly utilizing Informatica Data Quality. - Experience testing ETL solutions, particularly utilizing Informatica DVO product. - Experience using Enterprise Architecture modeling tools, frameworks and methodologies such as TOGAF, SPARX EA, UML, BPMN, ArchiMate etc. - Experience Designing Data Warehouses, applying Kimball’s Dimensional Modeling Techniques, Architecting ETL systems. - Experience working in Agile development software methodology, authoring technical documents / specifications. - Experience debugging complex software systems. - Significant experience implementing data-driven systems with large-scale relational or object-oriented databases. - Ability to work in a high pressure, result/deadline-oriented environment. Education: - Bachelor's Degree in technical discipline (Math’s, Science, Engineering, Computing, etc.) or a related study, or equivalent project-related experience.
We are looking for a Data Engineer (Databricks) to support the preparation phase for AI implementation by ensuring data is well-structured, reliable, and ready for advanced use cases. The ideal candidate acts as a bridge between business and technology, combining strong data understanding with hands-on technical expertise. Start Date: ASAP Location: Remote in Czechia Language: English, Czech Contract Type: B2B Responsibilities: - Collaborate with business and technical stakeholders to understand data needs and requirements - Utilize Databricks for data processing and engineering tasks - Prepare, structure, and ensure high quality of data for AI implementation - Work hands-on with data, transforming and organizing it for further usage - Act as a bridge between business and technology, translating requirements into data solutions Requirements: - Proven experience as a Data Engineer or in a similar role - Hands-on experience with Databricks - Strong understanding of data and ability to work with it hands-on - Experience working closely with both business and technical stakeholders - Ability to bridge business needs with technical implementation
Description About the Role At NinjaOne, we're looking for a skilled Data Engineer to join our team and help drive the future of our data infrastructure. You'll play a critical role in building, maintaining, and scaling our systems to ensure smooth data flow, accuracy, and security across the organization. This is an exciting opportunity to work on innovative projects, collaborate with cross-functional teams, and help shape how we leverage data to fuel growth, optimize products, and drive business decisions. Location - We are flexible on remote working from home, if you are located in the USA and reside in one of the following states - CA , CO , CT , FL , GA , *IL , KS , MA , MD, ME , NJ , NC , NY , OR , TN , TX , VA , and WA . We have physical offices in Austin, TX and Tampa, FL, if you prefer a hybrid option. We hire the best software engineers, but experience in our stack can't hurt: NinjaOne is built on Java , Kotlin , C++ , Golang and Postgres ; supporting millions of user endpoints and running as a scalable cloud service in AWS . Knowing large-scale datastore bottlenecks, asynchronous application design and client-server architecture will help you. What You'll be Doing - Data Pipeline Development: Design and implement scalable data pipelines that move and transform large volumes of data from multiple sources to central data warehouses , transforming data to enable business reporting and advanced analytics . - Database Management: Manage and optimize the performance of relational databases, ensuring data availability, reliability, and consistency. - Automation & Optimization: Automate and optimize data workflows to reduce manual processes and improve efficiency in data collection, storage, and processing. - Monitoring & Maintenance: Ensure the integrity and security of data across systems, monitor performance, and troubleshoot any issues that arise within the data pipeline. - Data Visualization: Build d ashboards and reports in Tableau and Databricks to expose key data points and trends to business stakeholders . - Collaboration: Work closely with data scientists, analysts, and other teams to gather requirements, understand data needs, and provide solutions that support data-driven decision-making. - Other duties as needed . About You - Bachelor's degree in Computer Science , Computer Engineering, Information Technology or equivalent work experience preferred. - 3 + years of experience in software development, with a strong focus on data engineering and data science. - E xperience in building data pipelines and managing large-scale data systems using technologies like SQL and Python - P roficiency in cloud platforms like AWS, GCP, or Azure, and experience with tools like Airflow , Kafka or dbt for orchestrating data workflows. - Experience with both relational databases inc luding MySQL, PostgreSQL and NoSQL databases like MongoDB, Cassandra. - E xperience with data warehousing concepts and tools suc h as Redshift, BigQuery , Snowflake. About Us NinjaOne automates the hardest parts of IT to deliver visibility, security, and control over all endpoints for more than 20,000 customers. The NinjaOne automated endpoint management platform is proven to increase productivity, reduce security risk, and lower costs for IT teams and managed service providers. NinjaOne is obsessed with customer success and provides free and unlimited onboarding, training, and support. NinjaOne is #1 on G2 in endpoint management, patch management, remote monitoring and management, and mobile device management. What You'll Love We are a collaborative, kind, and curious community. We honor your flexibility needs with full-time work that is hybrid remote. We have you covered with our comprehensive benefits package, which includes medical, dental, and vision insurance. We help you prepare for your financial future with our 401(k) plan. We prioritize your work-life balance with our unlimited PTO. We reward your work with opportunity for growth and advancement. Additional Information This position is NOT eligible for Visa sponsorship. Due to federal government security requirements associated with our FedRAMP-authorized environment, candidates must be U.S. citizens or lawful permanent residents. *Due to operational policies, NinjaOne is unable to hire for this role within the city limits of Chicago. We will consider all qualified candidates who reside outside of the city proper or are willing to self- relocate. Starting pay for the successful applicant depends on a variety of job-related factors, including but not limited to location, market demands, experience, job-related knowledge, and skills. The benefits available for this position include medical, dental, vision, 401(k) plan, life insurance coverage and PTO. For roles based in California, Colorado, Maryland, New Jersey, or Washington the base salary hiring range for this position is $ 90 ,000 to $ 1 7 0,000 per year. For roles based in New York, the base salary hiring range for this position is $ 9 0,000 to $ 1 7 0,000 per year. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, genetic information, marital status, veteran status, or any other status protected by applicable law. We are committed to providing an inclusive and diverse work environment. #LI- KS 2 #LI-Remote #BI-Remote #BI-Hybrid
Role Description We are looking for a Backend AI & Data Pipeline Engineer to own the end-to-end data processing infrastructure that powers Yuzee's intelligent course and job matching platform. You will design and maintain scalable, event-driven pipelines that process tens of thousands of daily records, generate semantic embeddings, and feed a growing knowledge graph used for personalised career pathway recommendations. What you'll do - Design and maintain three distinct processing pipelines — scheduled job ingestion, event-driven course processing, and a periodic knowledge graph builder — each with independent trigger logic and cost controls. - Generate and manage semantic embeddings via Amazon Bedrock (Titan v2), index them in MongoDB Atlas Vector Search, and calibrate similarity thresholds to ensure match accuracy. - Build and maintain a knowledge graph linking jobs, courses, skills, and industries using FP-Growth association rules and archetype-to-SOC code mapping. - Build and improve a two-stage discovery and matching API on AWS Lambda — vector retrieval first, then deep eligibility scoring with LLM re-ranking. - Right-size Fargate Spot instances and design resumable processing loops that tolerate interruption, keeping infrastructure costs under control as data volume scales. - Maintain and improve daily job scrapers across multiple sources and build institution data scrapers with robust HTML cleaning pipelines. Qualifications - 1+ years of backend engineering experience focused on data pipelines, ML infrastructure, or search systems. - Hands-on experience with AWS serverless and container services — Lambda, ECS Fargate, EventBridge, and Step Functions. - Strong Python skills — Pandas, async processing, bulk database operations, and text cleaning. - Familiarity with vector databases and semantic similarity search; MongoDB Atlas Vector Search experience is a strong plus. - Cost-conscious infrastructure mindset — you think in per-record compute costs, free tiers, Spot resilience, and right-sizing. - Ability to document and communicate complex architecture clearly to both technical and non-technical stakeholders. Requirements - Degree or existing proven experience. Benefits - You can work from home for the whole internship period. - A reference letter can be requested upon completion of internship. - A bit of flexibility with working time aside from the usual 9am to 6pm (Ex. 8am to 5pm / 7:30am to 4:30pm). - The possibility of retainment for part-time or full-time work post-internship based on your performance, even if you are not based in Malaysia.


