Job Closed

This listing is no longer active.

Livestock Information Ltd logo
Livestock Information Ltd

Maintaining livestock traceability for the benefit of government, industry and the economy

Data Engineer

Data EngineerData EngineerFull TimeRemoteMid LevelTeam 51-200Since 2019H1B No SponsorCompany SiteLinkedIn

Location

United Kingdom

Posted

59 days ago

Salary

£50K / year

Seniority

Mid Level

Job Description

Data Engineer

Livestock Information Ltd

The salary for this role is £50,000. This role is being offered on a 12 Month Fixed Term Contract. Are you an experienced Azure Data Engineer with a passion for building scalable, high-quality data solutions? Can you design and optimise data pipelines using tools like Azure Databricks and Data Factory to deliver real business impact? Do you enjoy collaborating with cross-functional teams to turn complex data into valuable insights If so, we’d love to hear from you! At Livestock Information Ltd, our mission is to be an outstanding service provider of livestock data, empowering government, industry, and farmers to effectively manage disease and infection control, whilst enhancing productivity and innovation in agriculture. To achieve this, we need great people, who share in our values. As a Data Engineer at LI, you will play a key role in delivering scalable, high-quality data solutions that support critical business needs. In this role, you’ll design and build new data services, lead migrations of existing solutions, and ensure the ongoing performance and enhancement of live data platforms. Working in a collaborative, agile environment, you’ll partner with cross-functional teams including Product Owners, Business Analysts, Developers, and Testers to deliver impactful data products. You’ll work with a modern Azure data stack, including Databricks, Synapse, Data Factory, and Data Lake technologies, using tools like Azure DevOps to support end-to-end development and delivery. Responsibilities Your responsibilities will include - Designing & Implementing Data Storage Solutions on Azure: Select Azure data storage services appropriately and design data storage schemas, ensuring they are optimised for specific use cases. - Building and Maintaining Data Pipelines for Data Integration and Processing: Build and maintain data pipelines for data movement and process orchestration. Collect data from various sources, transform data into a practical format and load it into appropriate data storage solutions on Azure using tools such as Azure Data Factory and Azure Databricks. Ensure data pipelines are scalable and efficient. - Ensuring Data Quality & Accuracy Through Testing & Validation: Test and verify data at different phases of the data pipeline, from extraction through to loading. - Optimising Data Processing Performance Through Tuning & Monitoring: Ensure data processing is efficient and scalable by tuning the data pipeline and monitoring performance; identify and resolve bottlenecks in the data pipeline and optimise data processing algorithms. - Developing & Maintaining Data Models & Schemas: Develop and maintain data models and schemas optimised for specific use cases. Select the appropriate data modelling techniques and ensure the data schema is scalable and efficient. - Collaborating With Other Teams to Provide Data for Analytics & Reporting: Work closely with other teams to provide data they need to perform their job functions. - Ensuring Data Security & Privacy Prerequisites Are Followed: Ensure that data streamed and processed on Azure confirms with applicable data security and privacy requirements such as the General Data Protection Regulation (GDPR). - Working with Large Datasets & Performing Data Analysis: Work with large datasets of hundreds of millions and billions of rows of data and perform data analysis - Staying updated on the latest Azure features, updates and best practices to ensure they are adopted effectively as appropriate You may be required to undertake other duties assigned by your line manager as appropriate

Job Requirements

  • We are looking for someone with the following
  • Proven experience in Python programming for data engineering and analytics
  • Strong proficiency in SQL, including the Microsoft T-SQL dialect
  • Hands-on experience with Azure data storage solutions
  • Practical knowledge of data integration and processing frameworks, such as Azure Databricks and Azure Data Factory
  • Solid understanding of data modelling concepts and schema design best practices
  • Strong analytical and problem-solving skills, with excellent attention to detail
  • Effective communication skills and the ability to work collaboratively within cross-functional teams.
  • For further information, please see the attached Job Description - Data Engineer Job Description

Benefits

  • As an LI employee, you will be entitled to our company benefits which include
  • A fully remote and flexible working set up
  • 25 days annual leave, increasing by 1 day each year of service up to maximum of 30 days (plus Bank Holidays)
  • 1 Privilege Day
  • 4 x annual salary Life Assurance
  • Market leading pension scheme through Legal & General
  • Enhanced Family Friendly Policies
  • Excellent learning, training, and career development opportunities
  • 24/7 access to our Employee Assistance Programme
  • A diverse and inclusive culture where everyone is respected and valued
  • Application and Interview Process
  • To apply for this role, you will need to submit a CV and answer some pre-screening questions.
  • The interview and assessment process for this role will involve a telephone interview with our Talent Team, followed by a first stage and a final stage interview. Interviews will be conducted either remotely via MS Teams or face to face.
  • If we need to make a reasonable adjustment so that you can complete your application, please contact us as soon as possible by emailing talent@livestockinformation.org.uk before the closing date to discuss your needs.
  • The closing date for this role is 4pm on Friday, 17th April. Applications received after the closing date will not be considered. Please note that we may close the advert early if we receive a high volume of applications
  • Artificial Intelligence
  • Artificial intelligence tools can be used to help you prepare your application. However, all statements and examples provided during the recruitment process should be accurate and based on your own knowledge and experience. Applications may be withdrawn if content is found to be plagiarised, including where ideas or experiences generated by artificial intelligence are presented as personal work. For further information, please click on the following link : AI & Recruitment at LI
  • Values
  • We approach our work with TACT; we are Transparent, Accountable, Committed and we work as a Team.
  • We are transparent together, embracing honesty and building trust. We are accountable to the industry and strive for the best possible solutions for the communities we serve. We drive change with passion, commitment and enthusiasm to deliver a great service to the livestock industry. We work kindly and cohesively which allows us to support each other in our personal and strategic goals. For more information on our values and what its like to work at Livestock Information, please visit our Careers Page : Careers - Livestock Information Ltd
  • At Livestock Information Ltd, we have a zero-tolerance approach to harassment, abuse, discrimination, or bullying during our recruitment process. Any such behaviour may result in a candidate being withdrawn from consideration We are committed to upholding our company values and ensuring a safe and respectful experience for all candidates and staff.
  • Diversity statement
  • At Livestock Information, we value diversity and believe that every individual brings different strengths. We understand that not everyone may possess every single skill outlined in this job description, but if you believe you have a solid skill set that enables you to excel in this position then we really encourage you to apply.
  • Further information
  • If you feel your application has not been treated in a fair and open manner or if you are dis-satisfied with any of your interactions during the recruitment process and you wish to make a complaint, you should contact our Talent team via email: talent@livestockinformation.org.uk
  • Disclaimer
  • Livestock Information Ltd. cannot sponsor work visas at this time.
  • Livestock Information Ltd. Does not discriminate based on race, sex, colour, religion, age, national origin, marital status, disability, veteran status, genetic information, sexual orientation, gender identity or any other reason prohibited by law in provision of employment opportunities and benefits.
  • Livestock Information Ltd. Does not accept unsolicited CVs from any source other than directly from a candidate. Any unsolicited CV’s not originating directly from a candidate and sent to Livestock Information Ltd or any Livestock Information Ltd employees, including unsolicited CVs sent to a Livestock Information Ltd mailing address or email address, are rejected. Livestock Information Ltd will NOT pay a fee for any placement resulting from the receipt of an unsolicited CV. Livestock Information Ltd considers any candidate for whom the recruiter has submitted an unsolicited CV referred by the company free of any fees.

Related Categories

Related Job Pages

More Data Engineer Jobs

DataForce by TransPerfect logo

Language Data Annotator

DataForce by TransPerfect

DataForce by TransPerfect is part of the TransPerfect family of companies, the world’s largest provider of language and technology solutions for global business, with offices in more than 100 cities worldwide. We offer high-quality data for Human-Machine Interaction to some of the most prestigious technology companies in the world. Our department focuses on gathering, enriching, and processing data for Machine Learning in different AI domains. To learn more about DataForce please visit us at https://www.transperfect.com/dataforce . For more information on the TransPerfect Family of Companies, please visit our website at www.transperfect.com .

Data Engineer59 days ago
Full TimeRemoteTeam 51-200

Role Description Data Force by TransPerfect is currently looking for English Language Data Annotator. This is an exciting opportunity to be part of an innovative project to improve Artificial Intelligence and technology. This role requires a high level of independence, adaptation, and accuracy - love of attention to detail will help to succeed in this role! - The Language Data Annotator will analyze data in the English and identify potential issues. - This data will be used to develop and improve the speech recognition capabilities of Artificial Intelligence systems. Qualifications - Excellent listening, writing, and spelling skills in English - Ability to meet quality standards - Familiarity with iOS software - Attention to detail - Flexibility, ability to adapt and react to changing priorities quickly and efficiently - Ability to track, analyze, and report issues - Problem-solving skills - Excellent time management - Minimum Bachelor's degree or its equivalent Company Description DataForce by TransPerfect is part of the TransPerfect family of companies, the world's largest provider of language and technology solutions for global business, with offices in more than 100 cities worldwide. - We offer high-quality data for Human-Machine Interaction to some of the most prestigious technology companies in the world. - Our department focuses on gathering, enriching and processing data for Machine Learning in different AI domains. To learn more about DataForce please visit us at https://www.transperfect.com/dataforce .

Philippines
Job Closed
National Guardian Life Insurance Company logo

Senior Data Engineer – Temp to Hire

National Guardian Life Insurance Company

We help people face life's financial challenges with confidence, dignity and grace.

Data Engineer59 days ago
Full TimeRemoteTeam 201-500Since 1909H1B No Sponsor

• The Senior Data Engineer works in an Agile team environment and is responsible for developing and optimizing our data and data pipeline architecture. • Optimizes data flow and collection for cross-functional teams and provides thought leadership to support software developers, database architects, data analysts and data scientists on data initiatives. • Ensures optimal data delivery architecture is consistent throughout ongoing projects. • Assists with the development and maintenance of analytic tools, data visualizations, and the platforms that supply them. • Trains and develops more junior engineers as appropriate and develops an understanding of the Company's strategic business goals, processes and solution requirements. • Assembles large, complex data sets that meet functional / non-functional business requirements. • Identifies, designs, and implements internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. • Implements infrastructure as code (IaC) practices to automate deployment and management of data services. • Contributes to the continuous integration and continuous deployment (CiCd) process. • Builds analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.

Wisconsin
Job Closed
RightsHelper logo

Senior Data Engineer

RightsHelper

RightsHelper is a ticketing analytics and operations platform. Our team includes operators, data scientists, and engineers who have worked inside professional sports organizations and understand the complexity of managing ticketing businesses at scale. We partner with teams and rightsholders across sports and live entertainment to support smarter, more confident decisions—without compromising flexibility, transparency, or control.

Data Engineer59 days ago

Summary We are looking for a hands-on Senior Data Engineer with experience building scalable ETL and data platforms in modern cloud environments. In this role, you will help design and implement the next generation of our data pipelines, enabling our platform to process and analyze growing volumes of ticketing, market, and consumer data. Our existing pipelines contain significant domain expertise and business logic that power our analytics platform. You will work to translate and evolve this logic into a scalable, maintainable architecture using modern engineering practices. This is a high-impact role with significant ownership over how data moves through our platform and supports the machine learning systems that drive our products. Job Function - Design and implement the next generation of our core data pipelines - Translate existing pipeline logic into a clean, maintainable architecture - Develop data pipelines in Python using modern best practices - Improve pipeline observability, logging, and failure recovery - Work with AWS services such as S3, Lambda, ECS, RDS, and/or Redshift - Design and implement data validation, monitoring, and alerting - Collaborate closely with leadership to define the future data architecture - Document pipeline design and operational workflows Our Tech Stack - Python - Apache Airflow - AWS (S3, ECS, Lambda, RDS, Redshift) - Docker - PostgreSQL What Success Looks Like - By the end of the engagement, you will have helped us: - Deliver the next generation of our core ETL pipelines with a scalable and maintainable architecture - Implement robust data validation, monitoring and observability across the data platform - Enhance pipeline reliability, performance and operational visibility

United States
Full TimeRemoteTeam 5,001-10,000

Who are Inchcape? At Inchcape, our vision is to have a connected world in which our customers trade successfully and make better decisions in every port, everywhere. We use technology and our global network to help our partners connect to a smoother, smarter ocean. Inchcape combines its worldwide infrastructure with local expertise through our global network of over 250 proprietary offices across 70 countries and a team of more than 3,000 committed professionals. Our diverse global customer base includes owners and charterers in the oil, cruise, container and bulk commodity sectors as well as naval, government and intergovernmental organizations. We have an ambitious growth model, and a career here is certainly going to be a rewarding one that will allow you to bring your skills & experience. We embrace change and are open to new thinking and pushing for positive change in our industry. Contract type: Fixed Term Contract (6 Months) Location: India Main Purpose of Job: This role will work closely work with the Data Analyst in the World of Ports team. Providing data collection, cleansing, verifying and updating in business intelligence system for global port, terminal and berths . This role will provide support for a global team and will need to able to ensure projects are completed while working with a dynamic and collaborative team. Each member of our team must understand the importance of accurate information and focus on gathering and distribution of appropriate and relevant data. Key Accountabilities : - Entering Global Port, Terminal & Berth information from source documents within allotted time limits. - Compiling, verifying accuracy and sorting information collected from the network - Understanding, interpreting, and capturing data from global network - Compile data for queries for information and access relevant files - Creating reports & maintain tracking sheet based on their allotted tasks Key Deliverable : Port Terminal and berth Information / Port Cost / Performance - Collect port data (General information, parameters, restrictions, local regulations, cargo work, navigation etc.) from various sources including but not limited to Local agents, published documents, Port websites and any other available open sources - Analyze the information, Vet the data and add / update the business intelligence system. - Act as the key point of contact and person responsible for assigned clients for providing the information requested. Port call and Commodity Movements - Update the vessel port call (vessel calls in ports) in the system populated by AIS by using mails, websites and other open sources of data, with special consideration to the cargo, quantity and stake holder information’s - update the commodity flow module by analyzing and updating the information from various sources like the UNCTAD and different customs websites (open sources) - Prepare periodic reports for clients and internal stake holders - Co-ordinate with clients for their market information and use available resources internally and externally for obtain the same Shipping News - Collect and update shipping market news, trends, key events etc. from both internal and external sources - Provide the stake holders with reports and vital trends to help the commercial team and external customers with making business decisions Skills and Experience Requirements : Skills Essential - Excellent Communication skills - Typing speed and accuracy - Good writing and rephrasing skills - Strong knowledge of Microsoft Office Tools - Ability to compare and check data for accuracy and consistency across different sources - Organization skills, with an ability to stay focused on assigned tasks Desirable - Strong in Advanced formulas in Microsoft Excel - Experience in working directly with Microsoft Excel on VBA Macros Experience Essential - Proven data entry work experience - Strong in Data Collection & Data Management Desirable - Expericence in shipping industry with the backend process - Basic knowledge of shipping terms - Expericence in ERP Tools Education and Professional Qualifications : Essential: - Any degree or equivalent business qualification. - Extensive knowledge of MS office products. Desirable: - A degree in Maritime Studies, Logistics, or similar field. - Certification in Microsoft Tools Why Inchcape Shipping Services? We believe in building a diverse and high-performing workforce that works together to provide our customers with the exceptional service they deserve. To reach the highest standards, we depend on our people, their welfare, training and expertise. We realize the value of our staff and know that your unique experiences, skills and passions will help you to build a rich and rewarding career in our dynamic industry. Our values are at the centre of everything we do, and the successful candidate will be expected to demonstrate and fully adopt these: Global Perspective - we connect the world and see the bigger picture. The Power of People - we rely on the strength of local agent knowledge and relationships. Progress - we adopt new thinking and push for positive change in our industry. In return, we will offer you: A diverse range of benefits, depending on your geographical location. Globally, as a minimum, you will receive: • A retirement savings plan • Life assurance • Holiday • Competitive salary • Employee Assistance Programme • Career development and progression opportunities • Values Award celebration events Plus, much more! If you think that you have the right skills, knowledge and attitude to join our team, then we invite you to apply by simply clicking at the top of this advert! #ISSthebiggerpicture Appointment to this role will be subject to satisfactory references and possession of a valid Right to Work documentation depending upon your geographical location. Inchcape is an Equal Employment Opportunity Employer and equality, diversity and inclusion are at the centre of everything we do. We recognise that our customers, colleagues, contractors and other partners are central to our success and that we work in a diverse society. We acknowledge not only our legal, but also our moral responsibility to ensure all groups have equality of opportunity. #LI-Remote Location India (Remote) Department Service Delivery - World Of Ports Employment Type India - Full Time Minimum Experience Entry-level

India
Job Closed