Atlassian is a publicly-traded computer software business specializing in collaboration, development, and issue-tracking software for teams. As an employer, Atlassian maintains a t
Principal Data Engineer
Location
Worldwide
Posted
1 day ago
Salary
0
Seniority
Lead
Job Description
Principal Data Engineer
Atlassian
Role Description Atlassian is looking for a Principal Data Engineer to join our Data Engineering Team and play a tech lead & architect role to build world-class data solutions and applications that power crucial business decisions throughout the organization. We are looking for an open-minded, structured thinker who is passionate about building systems at scale. You will enable a world-class engineering practice, drive the approach with which we use data, develop backend systems and data models to serve the needs of insights and play an active role in building Atlassian's data-driven culture. - Own the technical evolution of the data engineering capabilities and be responsible for ensuring solutions are being delivered incrementally, meeting outcomes, and promptly escalating risks and issues. - Establish a deep understanding of how things work in data engineering, use this to direct and coordinate the technical aspects of work across data engineering, and systematically improve productivity across the teams. - Maintain a high bar for operational data quality and proactively address performance, scale, complexity and security considerations. - Drive complex decisions that can impact the work in data engineering. Set the technical direction and balance customer and business needs with long-term maintainability & scale. - Understand and define the problem space, and architect solutions. Coordinate a team of engineers towards implementing them, unblocking them along the way if necessary. - Lead a team of data engineers through mentoring and coaching, work closely with the engineering manager, and provide consistent feedback to help them manage and grow the team. - Work with close counterparts in other departments as part of a multi-functional team, and build this culture in your team. Qualifications - You have 12+ years of experience in a Data Engineer role as an individual contributor. - You have at least 2 years of experience as a tech lead for a Data Engineering team. - You are an engineer with a track record of driving and delivering large (multi-person or multi-team) and complex efforts. - You are a great communicator and maintain many of the essential cross-team and cross-functional relationships necessary for the team's success. - Experience with building streaming pipelines with a micro-services architecture for low-latency analytics. - Experience working with varied forms of data infrastructure, including relational databases (e.g. SQL), Spark, and column stores (e.g. Redshift). - Experience building scalable data pipelines using Spark using Airflow scheduler/executor framework or similar scheduling tools. - Experience working in a technical environment with the latest technologies like AWS data services (Redshift, Athena, EMR) or similar Apache projects (Spark, Flink, Hive, or Kafka). - Understanding of Data Engineering tools/frameworks and standards to improve the productivity and quality of output for Data Engineers across the team. - Industry experience working with large-scale, high-performance data processing systems (batch and streaming) with a "Streaming First" mindset to drive Atlassian's business growth and improve the product experience. Benefits - Atlassian offers a wide range of perks and benefits designed to support you, your family and to help you engage with your local community. - Our offerings include health and wellbeing resources, paid volunteer days, and so much more. Company Description At Atlassian, we're motivated by a common goal: to unleash the potential of every team. Our software products help teams all over the planet and our solutions are designed for all types of work. Team collaboration through our tools makes what may be impossible alone, possible together. We believe that the unique contributions of all Atlassians create our success. To ensure that our products and culture continue to incorporate everyone's perspectives and experience, we never discriminate based on race, religion, national origin, gender identity or expression, sexual orientation, age, or marital, veteran, or disability status. All your information will be kept confidential according to EEO guidelines. To provide you the best experience, we can support with accommodations or adjustments at any stage of the recruitment process. Simply inform our Recruitment team during your conversation with them. To learn more about our culture and hiring process, visit go.atlassian.com/crh.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Technical Lead for Data
Peraton CorporationPeraton Corporation, a national security company headquartered in Herndon, Virginia, supplies solutions for mission-critical programs and systems. Founded in 2017, Peraton's missio
Role Description We are looking for a highly experienced Senior Technical Lead for Data to lead engineering and architectural efforts supporting the design, development, and maintenance of large-scale enterprise data systems. The ideal candidate will have deep expertise in data systems architecture, cloud environments, and enterprise database products, with proven leadership managing high-availability mission-critical systems in federal environments. You will oversee engineering teams, manage risk, develop technical performance metrics, and ensure the stable, secure, and efficient operation of data systems integrated with Identity and Access Management (IAM) and cloud platforms. As the Senior Technical Lead for Data, you will serve as the authoritative technical voice for all data-related workstreams, translating complex business and mission requirements into scalable, secure, and compliant data architectures while directly mentoring and guiding a team of data engineers. Sitting at the intersection of data engineering, cloud architecture, data governance, and technical leadership, you will collaborate closely with IAM engineers, platform architects, data scientists, security teams, and program leadership to ensure the data platform is modern, resilient, and aligned with federal data governance mandates and Zero Trust principles. - Lead the design, development, evaluation, testing, and integration of organizational data systems infrastructure, including both hardware and software components. - Ensure all systems meet stability, integrity, and efficient operation requirements in accordance with approved specifications; verify systems are properly maintained and optimized once operational. - Provide technical support for acquisition documentation, milestone reviews, and key operational and test events throughout the program lifecycle. - Oversee the systems engineering team and all engineering activities supporting organizational data and information systems, including networked software and associated operating systems. - Execute and manage risk assessments, develop mitigation strategies, and plan cost, schedule, and execution plans for assigned projects. - Develop and review technical performance metrics to track and communicate system health, delivery progress, and operational readiness. - Plan, manage, and coordinate multiple concurrent tasks, each with strict deadlines across a large-scale, highly available system integrated with business-critical platforms. - Drive continuous improvements and optimizations to database systems, data encryption, storage, and retention strategies. - Collaborate with IAM engineers, platform architects, data scientists, security teams, and program leadership to ensure data solutions align with organizational goals, AWS GovCloud architecture, and IAM frameworks. - Support agile development teams by assessing how enhancements, defects, and other requests impact the architecture and performance of the application. - Demonstrate expert knowledge of recent technological advances in computer science and engineering to inform architectural decisions and modernization efforts. - Provide excellent written and verbal communication to stakeholders at all levels, including executive briefings, technical documentation, and team guidance. Qualifications - Minimum of 12 years with BS/BA or 16 years with a HS diploma/equivalent. - Minimum 10 years of experience in Information Technology supporting complex data systems. - 7+ years of experience working with IBM, Oracle, and cloud-based database products. - Training and certification (mandatory) in target ICAM database products preferred, especially Oracle and DB2. - Strong understanding of software development, enhancement, and maintenance lifecycles. - Demonstrated experience with the planning, management, leadership, and coordination of multiple concurrent tasks with strict deadlines for large-scale, highly available systems integrated with business-critical platforms. - Demonstrated expertise in data encryption, data storage, and retention policies in secure federal environments. - Proven ability to perform complex technical tasks in a highly secure environment. - Strong technical knowledge of IT systems, IAM, and AWS GovCloud services. - Experience with agile/scrum methodologies. - Expert knowledge of recent advances in computer science and engineering. - Must be able to obtain and maintain the required agency clearance. - Must be a US Citizen. Preferred Skills - Experience with federal IT programs, regulatory compliance, and federal acquisition documentation. - Familiarity with enterprise data governance and security frameworks (NIST, FISMA, FedRAMP). - Knowledge of cloud-native database and data warehousing technologies on AWS GovCloud. - Experience supporting ATO processes, System Security Plans (SSPs), and continuous monitoring activities. - Familiarity with Zero Trust Architecture principles and federal mandates (OMB M-22-09, EO 14028). Company Description Peraton is a next-generation national security company that drives missions of consequence spanning the globe and extending to the farthest reaches of the galaxy. As the world’s leading mission capability integrator and transformative enterprise IT provider, we deliver trusted, highly differentiated solutions and technologies to protect our nation and allies. Peraton operates at the critical nexus between traditional and nontraditional threats across all domains: land, sea, space, air, and cyberspace. The company serves as a valued partner to essential government agencies and supports every branch of the U.S. armed forces. Each day, our employees do the can’t be done by solving the most daunting challenges facing our customers. Target Salary Range $135,000 - $216,000. This represents the typical salary range for this position. Salary is determined by various factors, including but not limited to, the scope and responsibilities of the position, the individual’s experience, education, knowledge, skills, and competencies, as well as geographic location and business and contract considerations. Depending on the position, employees may be eligible for overtime, shift differential, and a discretionary bonus in addition to base pay. EEO EEO: Equal opportunity employer, including disability and protected veterans, or other characteristics protected by law.
Role Description We are looking for a motivated and technically solid L1 Data Engineer to join our growing Data & Analytics team. In this role, you will be responsible for designing, building, and maintaining the data architecture and infrastructure that supports our organization's data strategy. You will work hands-on to develop, test, and deploy reliable data solutions — ensuring pipelines are scalable, efficient, and aligned with business requirements. This is an ideal opportunity for a data professional who is eager to deepen their expertise in cloud-native data platforms, particularly within the Microsoft Azure and Databricks ecosystem, and who thrives in a collaborative, fast-paced environment. - Design, develop, and maintain scalable data pipelines and ETL/ELT workflows to support business intelligence and analytics use cases. - Build and optimize data ingestion processes using Azure Data Factory and Databricks, ensuring data quality and consistency across all layers of the data platform. - Transform and process large datasets using PySpark and Python, applying best practices for performance and maintainability. - Write and optimize complex SQL queries to support analytical reporting and data validation requirements. - Collaborate with data architects and senior engineers to implement and maintain data models aligned with organizational standards. - Monitor, troubleshoot, and resolve pipeline failures and data quality issues, applying root-cause analysis to prevent recurrence. - Contribute to documentation of data pipelines, data dictionaries, and engineering standards. - Support the team in exploring and evaluating new tools and approaches to continuously improve the data infrastructure. Qualifications - 2+ years of professional experience in a Data Engineering or closely related role. - Strong proficiency in Python for data processing, transformation, and automation tasks. - Hands-on experience with Pandas for data manipulation and PySpark for distributed data processing. - Practical experience with Databricks, including notebook development, clusters, and job orchestration. - Experience building and managing data pipelines with Azure Data Factory. - Working knowledge of Azure Synapse Analytics, particularly Spark pool integration. - Solid SQL skills, including query writing, optimization, and performance tuning. - Familiarity with data engineering principles including incremental loading, data lake architecture, and Delta Lake. - Understanding of data governance and security concepts within a cloud data platform. Requirements - Experience with SQL Server migration projects, including schema conversion and data movement. - Exposure to Terraform for Azure infrastructure provisioning and management. - Familiarity with CI/CD practices applied to data engineering workflows. - Experience with Delta Sharing or Lakehouse Federation concepts. Certification Requirement Candidates are expected to hold or be actively working toward the Databricks Certified Data Engineer Associate certification. This certification validates foundational knowledge across the following domains: - Databricks Lakehouse Platform architecture and capabilities - ETL and ELT workflows using Spark SQL and PySpark - Incremental data processing and structured streaming - Production pipeline development and orchestration - Data governance and security within the Databricks environment
• Responsible for designing, building, and maintaining the data architecture and infrastructure required for our organization's data needs • Develop, test, and deploy data solutions, ensuring they meet our organization's needs
Role Description As a Data Engineer, He/She will be responsible for designing, building, and maintaining the data architecture and infrastructure required for our organization's data needs. They will also be responsible for developing, testing, and deploying data solutions, ensuring that they meet our organization's needs. Qualifications - Experience working with Python for data analysis and processing. - Proficient in using the Pandas library for data manipulation and analysis. - Experience working with Pyspark for distributed data processing. - Familiar with Databricks for managing big data workloads. - Experience with Azure Synapse Spark for building and managing data pipelines. - Experience with Azure Data Factory for creating and managing data integration workflows. - Proficient in SQL and experienced in optimizing SQL queries for performance. - Experience migrating data from SQL Server to other platforms. - Bonus: Experience with Terraform for Azure. - Knowledge assessed in the Databricks certification: Databricks Certified Data Engineering Associate. - Knowledge assessed in the Databricks certification: Databricks Certified Data Engineering Professional. Company Description


