BigData DevOps Engineer
Location
Worldwide
Posted
13 days ago
Salary
0
Seniority
Mid Level
No structured requirement data.
Job Description
BigData DevOps Engineer
Experian
Role Description As an important aide to both the IT Infrastructure and Development teams, you will help support existing systems 24x7 and be responsible for administering current Big Data environments. You will manage BigData Group environments and will work with teammates to maintain working solutions for our big data tech stack. To support product development following the product roadmap for product maintenance and enhancement such that the quality of software deliverables maintains excellent customer relationships and increases the customer base. You will report to the Engineering Manager, Data Platforms. If you have the skills, we would love to talk to you! What you'll do: - Implement, support, and administer Hadoop and AWS EMR environments. - Maintain Terraform and IaC for provisioning and managing AWS resources. - Provide CI/CD solutions. - Automate deployment and operations of Big Data technologies using DevOps tools. - Onboard Hadoop users and manage Kerberos, HDFS, Hive, HBase, and Yarn access. - Implement security best practices across HBase, HDFS, Kafka, Hive, and related components. - Tune Hadoop clusters, MapReduce, Spark workloads, and optimize EMR for performance and cost. - Monitor group health, logs, storage, and perform proactive capacity management. - Collaborate with infrastructure, network, DB, application, and BI teams to ensure platform reliability. - Support upgrades and patching for EMR, HBase, Spark, and other data platforms. - Troubleshoot system issues and analyze CPU, memory, OS, storage, and network performance. - Deploy and manage Hadoop clusters, configure HA, manage nodes, schedule jobs, and perform backups. - Support integrations across cloud/on-prem networks and data platforms. - Configure and maintain tools like Sentry, Spark, Kafka, Oozie, Solr, MongoDB, DocumentDB, and ELK. - Integrate AD/LDAP with Cloudera, manage Sentry policies. Qualifications - Typically requires a bachelor's degree (in Computer Science or related field) or equivalent. - 3+ years of Hadoop infrastructure administration – AWS EMR or Cloudera environments. - 2+ years of Linux (Redhat) system administration (Optional). - Cloud Platforms IaaS/PaaS – Cloud solutions: AWS, Azure, VMWare. - Gen AI / Agentic AI knowledge is a plus. - Automation skills – Ansible and Terraform. - Kerberos administration skills. - Experience coding languages SQL, Python. - Good experience in CI/CD tools such as Jenkins, Puppet, and Shell scripting. - Must have knowledge of DevOps tools. - Working knowledge of YARN, HBase, Hive, Spark, and Kafka. - Experience working with geographically distributed teams. - Bachelors or master's degree in Computer Science or equivalent experience. - Knowledge of our strategy and use of back-office applications. - To multi-lingual and multicultural environment, additional language skills are a bonus. - Receptive to change. - With our users at all levels. Benefits - Medical, life and dental insurance. - Asociacion Solidarista. - International Share Save Plan. - Flex Work/Work from home. - Paid time off. - Annual Performance Bonus. - Education Reimbursement. - Family Bonding. - Bereavement Leave. - Referral Program. - And more. This is a fully remote job opportunity. #LI-Remote Employee Status: Regular Role Type: Hybrid Department: Information Technology & Systems Schedule: Full Time
Related Guides
Related Categories
Related Job Pages
More DevOps Engineer Jobs
Lead Software/ DevOps Engineer
UnitedHealth GroupUnitedHealth Group is a healthcare and well-being company that’s dedicated to improving the health outcomes of millions around the world. We are comprised of two distinct and com
Role Description Join us to start Caring. Connecting. Growing together. You’ll enjoy the flexibility to work telecommute from anywhere within the U.S. as you take on some tough challenges. Willingness to work standard business hours in EST, CST, or MST required. - Collaborating with software developers, engineers, and operations teams - Monitoring sites and software to make sure they’re performing properly (including on-call shifts) - Anticipating potential problems before they occur (and coming up with solutions) - Conducting post-incident reviews - Documenting your work to turn findings into repeatable actions - Hands on Developer - Mentoring and coaching junior engineers - Conduct regular system audits and capacity planning exercises to identify areas for improvement and ensure readiness for future growth - Participate in on-call rotations and respond to incidents in a timely manner, ensuring quick resolution and effective communication with stakeholders - Establish and maintain best practices for monitoring, logging, and alerting using tools like Datadog, Prometheus, and Grafana - Configure and maintain services such as load balancers, relational & NoSQL databases, and messaging systems while ensuring high availability and performance - Design, develop, and deploy AI-powered solutions to address complex business challenges with emphasis on responsible use of AI Qualifications - Bachelor's degree in CS or IT or engineering related field - 10+ years of experience in object-oriented programming language JAVA - 5+ years of experience as a Lead Software Engineer, DevOps Engineer or in IT Operations - 3+ years of experience with any one public cloud platform like AWS or Azure or GCP - 2+ years of experience with container technologies like Docker and Kubernetes - 1+ years of experience with automation and scripting tools such as Python, Bash, PowerShell, and Perl Requirements - Excellent communication and interpersonal skills, with the ability to work collaboratively with development teams, stakeholders, and management - Experience in problem-solving skills on complex technical issues and a proactive attitude towards identifying and addressing potential issues - Experience with public cloud platforms, hybrid cloud environments, and migration strategies - Experience with REST API design, micro services, and event driven architecture - Experience with configuration and deployment management tools such as Ansible, Terraform - Experience with configuration and maintenance of services such as load balancers, relational & NoSQL databases, and messaging systems - Experience in monitoring and alerting tools such as Datadog, Prometheus, and Grafana - Experience with incident response and post-mortem analysis Benefits - Paid Time Off which you start to accrue with your first pay period plus 8 Paid Holidays - Medical Plan options along with participation in a Health Spending Account or a Health Saving account - Dental, Vision, Life & AD&D Insurance along with Short-term disability and Long-Term Disability coverage - 401(k) Savings Plan, Employee Stock Purchase Plan - Education Reimbursement - Employee Discounts - Employee Assistance Program - Employee Referral Bonus Program - Voluntary Benefits (pet insurance, legal insurance, LTC Insurance, etc.)
DevSecOps Consultant
Computer Task Group, IncCTG, a Cegeka company, is at the forefront of digital transformation, providing IT and business solutions that accelerate project momentum and deliver desired value. Over nearly 60 years, we have earned a reputation as a faster and more reliable, results-driven partner. Our vision is to be an indispensable partner to our clients and the preferred career destination for digital and technology experts. CTG leverages the expertise of over 9,000 team members in 19 countries to provide innovative solutions. Together, we operate across the Americas, Europe, and India, working in close cooperation with over 3,000 clients in many of today's highest-growth industries. For more information, visit www.ctg.com . Our culture is a direct result of the people who work at CTG, the values we hold, and the actions we take. In other words, our people define our culture. It's a living, breathing thing that is renewed every day through the ways we engage with each other, our clients, and our communities. Part of our mission is to cultivate a workplace that attracts and develops the best people. CTG will consider for employment all qualified applicants including those with criminal histories in a manner consistent with the requirements of all applicable local, state, and federal laws. CTG is an Equal Opportunity Employer. CTG will assure equal opportunity and consideration to all applicants and employees in recruitment, selection, placement, training, benefits, compensation, promotion, transfer, and release of individuals without regard to race, creed, religion, color, national origin, sex, sexual orientation, gender identity and gender expression, age, disability, marital or veteran status, citizenship status, or any other discriminatory factors as required by law. CTG is fully committed to promoting employment opportunities for members of protected classes.
Role Description CTG is seeking a highly skilled DevSecOps Consultant to support our client’s enterprise technology initiatives. This role is responsible for analyzing IT environments, designing DevSecOps automation strategies, implementing secure CI/CD solutions, and supporting cloud and infrastructure modernization efforts across development and operations teams. Location: Remote Duration: 7 months Key Responsibilities: - Assess client IT environments and develop DevSecOps transformation roadmaps. - Design and implement secure CI/CD pipelines and automation frameworks. - Architect DevSecOps tooling integrations across cloud and enterprise platforms. - Support full application lifecycle activities including build, test, deployment, monitoring, and operational support. - Integrate security controls, vulnerability scanning, compliance automation, and monitoring into DevOps processes. - Mentor DevOps Engineers and provide technical leadership on DevSecOps best practices. - Research emerging technologies and contribute to solution development initiatives. Qualifications - 7+ years of experience in DevOps, DevSecOps, cloud engineering, or infrastructure automation. - Bachelor’s degree in Computer Science, Information Technology, Engineering, or related field preferred. - Relevant cloud, DevOps, or security certifications are a plus. - Excellent verbal and written English communication skills and the ability to interact professionally with a diverse group are required. Requirements - Strong experience with DevSecOps methodologies, CI/CD pipelines, and automation tools. - Hands-on experience with Jenkins, GitLab CI/CD, Azure DevOps, GitHub Actions, or similar tools. - Cloud platform experience with AWS, Azure, or Google Cloud Platform. - Knowledge of Docker, Kubernetes, Terraform, Ansible, and Infrastructure as Code practices. - Experience integrating security tools such as SonarQube, Fortify, Checkmarx, or related platforms. - Scripting and automation experience using Python, Bash, or PowerShell. - Familiarity with Application Performance Management and monitoring tools. Benefits - The expected base salary for this position ranges from $50.00 to $55.00/hour. - Salary offers are based on a wide range of factors including relevant skills, training, experience, education, market factors, and where applicable, licensure or certifications obtained. - In addition to salary, a competitive benefit package is also offered. Company Description CTG, a Cegeka company, delivers IT and business solutions that enhance clients’ digital agility, empowering them to seize new opportunities and overcome any challenge. Backed by more than 60 years’ experience and a commitment to being a reliable, results-driven partner, we work shoulder to shoulder with clients to shape digital together. - Our vision is to be an indispensable partner to our clients and the preferred career destination for digital and technology experts. - With more than 9,000 team members in over 15 countries, we combine global expertise with local insight to deliver innovative solutions. - We operate across the Americas, Europe, and India, working with over 3,000 clients in many of today's highest-growth industries.
• Automação de processos de CI/CD • Gerência de configuração • Automação de provisionamento infraestrutura • Compreender os produtos da empresa bem como suas regras de negócio • Desenvolver os requisitos dos produtos com base na especificação • Reportar resultados de atividades bem como tirar dúvidas com o líder técnico e PO • Identificar, reportar e tratar dívidas técnicas • Analisar e corrigir problemas em ambiente produtivo.
Principal Site Reliability Engineer
SonicWallDelivering real-time breach detection and prevention solutions backed by SonicWall Capture Threat Network.
• Own the reliability, scalability, and operational excellence of our Cloud-based services. • Define and enforce reliability standards. • Drive the adoption of SRE practices across engineering teams. • Build the systems and tooling that keep our production infrastructure healthy. • Define, publish, and continuously refine Service Level Indicators (SLIs), Service Level Objectives (SLOs), and Service Level Agreements (SLAs) for all critical services, partnering with product and engineering leadership. • Own the error budget framework: track consumption, enforce error budget policies, and drive reliability investments when budgets are at risk. • Lead the design and implementation of comprehensive observability platforms — metrics, structured logging, and distributed tracing — to ensure full visibility into production systems. • Drive toil reduction initiatives by identifying and automating repetitive, manual operational work, targeting measurable reduction in operational burden across teams. • Design and execute chaos engineering programs to proactively uncover reliability weaknesses in our infrastructure and services before they impact customers. • Lead blameless postmortem culture: facilitate incident retrospectives, extract systemic learnings, and track corrective action items to completion. • Build and improve on-call incident response processes, runbooks, and escalation paths; manage and optimize on-call rotation health to prevent burnout. • Help design, build, and support infrastructure and security technologies within the cloud that offer resiliency, observability, and optimized cost. • Develop solutions for automated deployment of software and services on our production infrastructure hosted on AWS, applying reliability engineering principles throughout. • Shape how mission-critical enterprise software solutions are developed and deployed using optimized CI/CD pipelines that embed reliability and quality gates. • Develop management solutions for services across multiple cloud platforms and data centers, with a focus on fault tolerance and graceful degradation. • Collaborate with developers to bring new features and services into production using production-readiness reviews and launch checklists. • Champion reliability engineering best practices across the organization, embedding SRE principles into the software development lifecycle. • Mentor team members on SRE philosophy, technical decision-making, code reviews, and cloud engineering best practices. • Participate in roadmap planning, identify areas of improvement, and perform technology evaluation and selection.


