Texas State Library and Archives Commision

Proficiency with creating CI/CD pipelines and stages with Harness using YAML templates. Kong plugin development using Lua or Go. Experience with Package Managers like LuaRocks. Experience with containerization (Docker), and orchestration tools (Kubernetes). Experience with cloud platforms such as AWS, Google Cloud, or Azure. Familiarity with monitoring and logging tools like Datadog and OpenTelemetry.

Senior Engineer, Data Engineering

Location

United States

Posted

5 days ago

Salary

$70 / hour

Seniority

Senior

Job Description

Senior Engineer, Data Engineering

Texas State Library and Archives Commision

Role Description As a member of the team, you will be responsible for working in data platforms, both on-premises and on the cloud to help accelerate the Firm's transformation initiatives. As an expert data engineer with expertise in data platforms, you will be responsible for: - Configuring data pipeline solutions to access distributed data sources across on-premises, public cloud service providers, and SaaS ecosystems. - Working with a team of data engineers across global hubs to build target state platforms and migrate existing applications and products. - Building next-gen data platforms on the cloud to enable our Enterprise colleagues with rapid data access and incubate emerging technologies. Qualifications - 5 - 8 years of IT experience with about 4+ years in data engineering and management. - Bachelors' / Masters' degree from an accredited college/university in business related or technology related field. - Experience in all facets of software development life cycle like analysis, design, development, data conversion, data security, data quality, data integration and implementation. - Experience in implementing effective and successful on-premises and cloud-based Data Migration and Data Integration strategies across systems using industry standard tools. - Experience in Data Management, Architecture, Data Modeling and adopting best practices and policies in the Data Management space would be an added advantage. - Experience in leading and delivering data centric projects with concentration on Data Quality and adherence to data standards and best practices. - Experience building and coding applications using Python, Spark, Kafka, Storm, HDFS, Hbase, Hive, Sqoop, etc. - Relevant certifications in AWS or Azure preferred. - Experience with relational SQL and NoSQL databases, including Postgres and Cassandra. - Experience building data pipelines for enterprise-grade data and analytics platforms that are highly scalable, compliant and secure with robust data quality, data governance, data discovery, catalog and visualization capabilities. - Proficient in using Python, Java, or other high-performance languages, developer tools, CI/CD, DevOps, monitoring tools and engineering data migration solutions. - Collaborative - able to build strong relations that enable robust debate and survive periodic disagreements regarding priorities. - Ability to communicate effectively with technical and non-technical audiences. - Industry recognized cloud certifications, patents, open-source contributions are preferred. Company Description

Related Categories

Related Job Pages

More Data Engineer Jobs

Reply logo

Data Engineer

Reply

Reply designs and implements innovative solutions in the areas: Digital Services, Technology and Consulting.

Data Engineer5 days ago
Full TimeRemoteTeam 10,001+Since 1996H1B Sponsor

Role Description Atuação remota!! - Construir e gerenciar pipelines confiáveis de dados envolvendo ingestão/coleta, processamento, integração, armazenamento e disponibilização de dados na organização. - Atuar em uma arquitetura de sistemas distribuídos para o processamento de dados massivos em paralelo (MPP), combinando diversas fontes de dados heterogêneas e colaborando com equipes de análise e ciência de dados na construção de soluções e geração de valor baseadas em dados. Qualifications - Experiência prática com ingestão, integração, processamento e armazenamento de grandes volumes de dados. - Atuação em projetos de Big Data. - Behavior Driven Development (BDD). - Extração de dados em Python e processamento de dados com PySpark. - Experiências em ferramentas ETL's. - Conhecimento em modelagem de dados relacionais e dimensionais (Data WareHouse). - Experiência com bancos de dados SQL. - Experiência com conjunto de ferramentas relacionadas a Big Data na AWS como: EMR, Kinesis, RedShift, S3, Glue, ElasticSearch. - Conhecimento em Kafka. - Conhecimento com Data Lake e Data Ops. Requirements - Certificações AWS (desejável/diferencial). - Conhecimento em ferramentas de provisionamento de infraestrutura em cloud via código tais como: Terraform, CloudFormation (desejável/diferencial). Benefits - Cartão flexível Swile pra você usar como quiser (VA e VR). - Totalpass ou Gympass. - Apoio à Saúde Mental – Psicologia Viva. - Plano de Saúde Bradesco. - Plano Odontológico Bradesco. - Participação nos Lucros. - Auxílio-Creche para nossas mamães. - Incentivo a certificações. - Palestras e Webinars especiais. - Programa RAF de bonificação por indicações. - Seguro de Vida. - Subsídio para Inglês ou Italiano. - Desconto Open English. - Presente de aniversário. - Possibilidade de mudança do país. - Parcerias com Universidades.

Brazil
Globe Life AO logo

Online Data Entry Specialist

Globe Life AO

Work for a Fortune 500 company that rewards performance, invests in your growth, and provides a launchpad for a high-earning remote sales career. This isn’t just a job — it’s your path to leadership, income, and long-term success.

Data Engineer6 days ago
Full TimeRemoteTeam 51-200

Role Description We are actively hiring and scheduling interviews this week for a fully remote Work From Home position. Immediate hiring – secure your spot and get hired. Entry level position for the applicants and full training provided. This is a legitimate opportunity with full training provided and guidance to obtain your Life & Health Insurance license. No prior experience required. We are looking for motivated U.S. residents ready to grow in a long-term remote career. - Communicate professionally with clients - Provide information and guidance - Follow a structured system - Maintain consistent performance Qualifications - Strong communication skills - Reliable internet connection - Self-motivated and coachable - Must be a U.S. resident - Willingness to obtain a Life & Health Insurance license (assistance provided) Benefits - 100% Remote - Full training program - Licensing guidance and support - Advancement opportunities - Supportive leadership team Company Description

United States
Job Closed

Role Description Oxford's client, a global leader in the biopharma industry, is seeking a Data Engineer to support a growing workload within their innovation team. This role will be instrumental in building and maintaining scalable data pipelines, delivering high-quality, production-ready datasets to cross-functional teams including IT and AI. - Design, build, and maintain robust data pipelines within a Snowflake environment - Transform raw global data (including datasets from China and other international regions) into clean, structured, and production-ready outputs - Deliver polished datasets to downstream teams including IT and AI/ML teams - Collaborate closely with AI teams to support data-driven models and workflow automation initiatives - Utilize Databricks to support data processing, analytics, and pipeline orchestration - Structure and optimize data models to ensure usability, scalability, and performance - Partner with IT teams involved in data mining and analytics to ensure seamless data access and usability - Help reduce backlog by supporting and scaling existing data engineering efforts - Ensure data processes align with industry compliance standards and business requirements Qualifications - Exposure to AI/ML workflows, including automation and data pipeline integration - Experience in the biopharma, life sciences, or regulated industries - Proven experience as a Data Engineer in a complex, data-driven environment - Strong hands-on experience with Snowflake - Experience building and maintaining end-to-end data pipelines - Proficiency with Databricks and modern data engineering tools - Solid understanding of data modeling and structuring techniques - Experience working with both structured and semi-structured data - Ability to work independently as a self-starter in a fast-paced environment - Experience working with global datasets and distributed teams Preferred Qualifications - Familiarity with data compliance, governance, and regulatory requirements - Experience supporting or collaborating with AI/ML teams

EST (UTC-5)

Role Description Our client is looking for an AI / Data Engineer to design and deliver data platforms, pipelines and AI-enabled solutions for our clients. This is a hands-on consulting and delivery role for someone who can take ownership of ambiguous client problems, shape a practical technical approach, and deliver robust solutions across data engineering, data architecture and AI innovation. You will work with the company’s colleagues, client stakeholders and external technical teams to acquire, structure, transform and expose data through analytics, applications, APIs and AI-enabled experiences. Duties & Responsibilities - Own the delivery of client data and AI engineering work from discovery through design, implementation, testing and deployment. - Work with client stakeholders to understand business problems, clarify requirements and translate them into practical technical solutions. - Design and build data ingestion pipelines from APIs, third-party systems, files, online sources and operational platforms. - Develop scalable warehouse and transformation layers that convert raw data into trusted, reusable client data products. - Apply sound data architecture, modelling, quality, lineage and governance practices. - Identify and implement opportunities to use AI, automation, retrieval and agent-based workflows within client solutions. - Build or support APIs, data access layers and application integrations that make data products usable by reporting tools, software applications and AI experiences. - Contribute reusable technical patterns, documentation and engineering standards across the company. Qualifications - Ownership and Client Problem-Solving: You take ownership of outcomes rather than waiting for fully defined requirements. You are comfortable working through ambiguity, asking the right questions, identifying gaps and helping clients move from a business problem to a practical solution. - AI-First Mindset: You are interested in how AI can improve data collection, enrichment, research, automation, retrieval and decision support. You do not need to be an AI researcher, but you should be comfortable evaluating where LLMs, agents and AI-enabled workflows can create practical value. - Data Engineering and Warehousing Capability: You have strong hands-on experience designing and delivering modern data pipelines and warehouse solutions. You understand ingestion patterns, orchestration, data modelling, transformation layers, quality controls, observability, lineage, performance and security. - Collaborative Delivery Mindset: You work well in a consulting environment, communicate clearly with technical and non-technical stakeholders, and are comfortable sharing ideas, challenging assumptions and working closely with client and internal delivery teams. Requirements - Approximately five to ten years of relevant experience in data engineering, analytics engineering, data platform delivery or related technical consulting roles. - Practical experience with several of the following: - Modern cloud data warehouses such as Snowflake, BigQuery, Redshift, Databricks or Synapse. - SQL, data modelling and transformation frameworks such as dbt. - Cloud platforms, particularly AWS, Azure or GCP. - API integration, external data ingestion and pipeline orchestration. - Data quality, monitoring, observability and governance practices. - Software engineering practices including Git, automated testing and CI/CD. - Building data products for analytics, reporting, APIs or software applications. - Experience with AI orchestration frameworks, LLM tool calling, retrieval workflows, vector search, agent-based systems or related AI technologies. Success in This Role Success will mean delivering reliable, scalable and commercially useful data and AI solutions for the clients. You will combine sound engineering discipline with strong client problem-solving: understanding the real need, designing the right solution, delivering it effectively and helping clients derive measurable value from their data. Kindly regard your application as unsuccessful if you have not heard from the agency within 2 weeks.

Worldwide