For Agents. For Brands. For Success.
Principal Software Engineer, BI & Data Platform
Location
Canada
Posted
74 days ago
Salary
$120K / year
Seniority
Lead
Job Description
Principal Software Engineer, BI & Data Platform
Calabrio, Inc.
• Architect and develop large-scale, mission-critical BI and data platform solutions serving millions of users across the globe, leveraging AWS native technologies including Athena, Redshift, Glue, QuickSight, and S3. • Lead the design and implementation of robust data pipelines, data lakes, and data warehouses using modern architectures (Iceberg, Parquet, columnar formats) to support real-time and batch analytics at scale. • Drive technical strategy and architectural decisions for the BI platform, including data modeling, query optimization, performance tuning, and cost optimization across AWS services. • Build and maintain sophisticated back-end services, ETL/ELT workflows, and front-end analytics applications using Python, SQL, React, and modern web technologies. • Design and implement efficient data storage solutions across relational databases (Redshift, PostgreSQL) and non-relational databases (DynamoDB, S3), ensuring optimal performance and cost-efficiency. • Develop and maintain REST APIs and event-driven architectures to enable seamless integration between data services, analytics tools, and customer-facing applications. • Serve as the technical lead and mentor for engineering teams, conducting architecture reviews, code reviews, and providing guidance on complex technical challenges. • Collaborate with cross-functional teams including data engineers, analytics engineers, product managers, and DevOps to deliver innovative BI solutions that drive business value. • Champion engineering excellence by establishing best practices, design patterns, and coding standards for data-intensive applications. • Lead Agile ceremonies, drive sprint planning, and ensure timely delivery of high-quality software solutions while maintaining technical debt at manageable levels. • Evaluate and integrate emerging AWS services and open-source technologies to continuously improve platform capabilities and developer productivity. • Troubleshoot and resolve complex performance issues in distributed data systems, optimizing query performance, data processing workflows, and infrastructure costs. • Participate in strategic planning and roadmap development, translating business requirements into scalable technical solutions. • Contribute to the team on-call rotation, providing expert-level support for production environments and mentoring team members on incident response.
Job Requirements
- 10+ years of professional experience in software development, with at least 5 years focused on data engineering, business intelligence, or analytics platforms in enterprise SaaS environments.
- Deep expertise in AWS data and analytics services including **Athena, Redshift, Glue, S3, QuickSight, Lake Formation**, with hands-on experience architecting and operating production workloads.
- Advanced proficiency in **Python** for data processing, ETL/ELT development, and backend services, with strong knowledge of frameworks such as FastAPI, Flask, or similar.
- Expert-level **SQL** skills including complex query optimization, window functions, CTEs, and performance tuning across multiple database engines (Redshift, PostgreSQL, Athena).
- Strong full-stack development capabilities with proficiency in **React, JavaScript/TypeScript**, and modern front-end frameworks for building analytics dashboards and data visualization interfaces.
- Proven experience with **Apache Iceberg, Parquet, or similar columnar formats** and modern data lake architectures.
- Deep understanding of data modeling techniques for both OLTP and OLAP workloads, including dimensional modeling, star/snowflake schemas, and denormalization strategies.
- Strong, in-depth experience with AI coding assistants such as GitHub Copilot, Cursor, and Windsurf to accelerate development and improve code quality.
- Experience with both relational databases (Redshift, PostgreSQL, Aurora) and non-relational databases (DynamoDB, OpenSearch, DocumentDB).
- Demonstrated ability to design and implement scalable REST APIs, event-driven architectures (Lamda, EventBridge, SQS, SNS), and microservices patterns.
- Experience with **AWS CDK (Cloud Development Kit)** or CloudFormation/Terraform for infrastructure-as-code and automated deployment pipelines.
- Strong background in data pipeline orchestration using AWS Step Functions, Glue workflows, or similar tools.
- Proven track record of leading technical initiatives, mentoring senior engineers, and driving architectural decisions in complex distributed systems.
- Experience with CI/CD pipelines, automated testing frameworks (pytest, Playwright), and DevOps practices for data platforms.
- Excellent problem-solving skills with the ability to troubleshoot complex issues in large-scale, distributed data systems.
- Strong understanding of software development lifecycle (SDLC), Agile methodology, and experience leading technical teams through the full development lifecycle.
- Outstanding communication skills with the ability to articulate complex technical concepts to both technical and non-technical stakeholders.
- Adaptable with the ability to switch technical stacks and domains to respond to changing customer and business needs while maintaining strategic focus.
Benefits
- Global team recognized for their passion and innovation
- Innovative product culture and project exposure
- Training and development from industry-leading experts
- Cutting edge benefit programs that include: 401(k) with company matching; medical, dental, and vision insurance; disability and life insurance; flexible PTO; paid holidays and parental leave; tuition reimbursement and more
- We offer market competitive pay and benefits based upon the candidate’s skills, experience, and qualifications.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer
Capital Bank MarylandCapital Bank, established in 1999, is a publicly traded company with over $2.43 billion in assets. The bank operates five branches in the greater Washington, DC
• Serve as the expert for ETL and DB solutions, collaborating with business stakeholders and IT teams to define requirements, gather data, and implement optimized data solutions. • Design, implement, and maintain data systems in Snowflake to ensure data scalability and accessibility. • Implement and manage data lakes and data warehouses, creating pipelines and data models to enable efficient analytics and reporting. • Establish and document strategies for managing data transfer processes, including secure file transfers (SFTP), batch data processing, and real-time streaming. • Build and optimize ETL pipelines for data extraction, transformation, and loading into operational databases or analytical platforms. • Integrate and support data visualization tools such as Power BI, Sisense, Google Looker, Tableau, or similar platforms to enable actionable insights for business stakeholders. • Develop and maintain optimized data models for dashboards and reporting, ensuring compatibility with visualization tools. • Plan, coordinate, and implement database migrations, upgrades, and patches with minimal downtime. • Define and enforce database governance policies, including data integrity, security, and compliance with regulatory requirements. • Analyze and resolve database performance issues by optimizing queries, indexes, and schema designs. • Partner with vendors to evaluate, select, and implement database tools, services, and technologies; stay informed about product roadmaps and industry trends. • Develop disaster recovery and high-availability solutions, including replication, clustering, and failover.
• Design and implement data pipelines (ETL/ELT) using modern tools (e.g., Apache Airflow, DBT, Dataflow); • Integrate data from transactional systems, APIs, and relational and non-relational databases; • Create and maintain optimized data structures in analytical environments (data lakes and data warehouses); • Ensure data governance, data quality, and data cataloging; • Automate routines for data extraction, transformation, and loading; • Support data scientists, analysts, and product squads with reliable, well-modeled data; • Participate in modernization and data migration projects to the cloud; • Monitor and resolve failures in pipelines and other critical data processes.
• Supports ECS/EAD (Enterprise Case Selection/Enterprise Anomaly Detection) data engineering tasks including ingestion scripting, basic transformation logic, and data preparation activities for analytics and operational workloads. • Assists senior engineers in building and maintaining pipelines, validating data accuracy, and troubleshooting low‑complexity ingestion errors. • Performs initial schema mapping, data profiling, and documentation updates. • Works with analysts and data scientists to ensure datasets are complete, accessible, and aligned with ECS/EAD mission and compliance requirements
• Supports ECS/EAD (Enterprise Case Selection/Enterprise Anomaly Detection) data engineering efforts with a focus on automation and repeatable data processing workflows. • Builds automated data validation routines, ETL test harnesses, and monitoring scripts to ensure pipeline reliability, data integrity, and compliance with agency standards. • Implements ingestion and transformation components, integrates with cloud data services, and resolves pipeline defects through automated checks. • Collaborates with senior engineers, analysts, and cloud teams to ensure data flows are accurate, secure, and aligned with ECS/EAD modernization patterns.


