Protiviti

Protiviti is a global consulting firm that helps businesses solve critical challenges in finance, technology, operations, data, and governance. Established in 2

Data Engineering Senior Manager - Microsoft Fabric and Databricks

Location

New York

Posted

56 days ago

Salary

$159K - $255K / year

Seniority

Lead

Bachelor Degree

Job Description

Data Engineering Senior Manager - Microsoft Fabric and Databricks

Protiviti

Title: Data Engineering Senior Manager (Microsoft Fabric & Databricks) Location: NEW YORK CITY Job Description: You Belong Here The Protiviti Career provides opportunity to learn, inspire, and advance within a collaborative and inclusive culture.  We hire curious individuals for whom learning is a passion. We lean into our mission: We Care. We Collaborate. We Deliver. At every level, we champion leaders who live our values of integrity, inclusion, innovation, and commitment to success. Imagining our work as a journey, we believe integrity guides our way, inclusion moves us forward together, innovation creates new destinations, and our commitment to success empowers us to deliver on our vision to be the most trusted global consulting firm. Where We Need You Protiviti is looking for a Technology Consulting Senior Manager to join our growing Enterprise Data & Analytics team. What You Can Expect As a Senior Manager, you’ll partner with our clients to identify and manage risk. You'll develop strategies to solve complex business problems using new ways of thinking, and devise solutions to support your clients' needs through using the latest tools and methods.  You’ll develop lasting relationships with client personnel and further these relationships through quality product delivery.  You’ll foster a network within the business community and serve as an ambassador of Protiviti in the market. You will also be a mentor and provide performance development to teams as you oversee the successful completion of project work plans. What Will Help You Be Successful - You enjoy designing and delivering modern data engineering solutions with Microsoft Fabric and Azure Databricks. - You are motivated to learn and interested in all things related to business intelligence reporting and data engineering, including the latest trends and developments.  - You are passionate about building relationships with clients and providing exceptional experiences. - You have an inherent interest in project management and team leadership. - You promote a positive team culture that fosters open communication among all engagement team members.  - You create development opportunities for others, including participating in the creation and rollout of training, and ways for your team to improve our clients and communities. - You have interest in contributing to the preparation of client proposals and strategies to win new business. - You excel at identifying opportunities to integrate product solutions and resources to improve client service capabilities. - You have interest in working with a diverse portfolio of clients across industries. Do Your Talents Include the Following? You demonstrate Leadership & Management capabilities through the following:  - You care about the success and well-being of both your clients and colleagues.  - You collaborate by creating a team environment where everyone feels that they are valued, welcomed, and can contribute their ideas.  - You deliver and earn the trust of your clients and colleagues by building best-in-class solutions and managing deliverables.  You have experience with many and familiarity with most of the following Concepts and Methodologies:  - Ability to articulate the value of Microsoft Fabric and Business Intelligence solutions to customers and colleagues, and proven track record of data storytelling to guide decision-making.  - Participation in the process of formulating, communicating, and executing data strategies that align with organizational goals and objectives leveraging Microsoft technologies.  - Experience in assessing existing data solutions, whether on-premises or in the cloud, to inform improvement initiatives.  - Basic skills in architecting future state data solutions on Azure and Microsoft Fabric based on the Well-Architected Framework.  - Some experience in planning and migrating data solutions from various sources, including on-premises, other cloud platforms, or legacy systems, to Microsoft Cloud 1st Solutions.  - Proficiency with implementing ETL processes and databases on Microsoft Technologies.  - Experience optimizing existing data implementations, including pipelines, queries, and data models, to enhance performance and delivery of value to the business.  - Knowledge of administering and maintaining data solutions to ensure stability, security, scalability, and affordability.  - Proficiency in data engineering, including data movement, data integration, transformation, and cleansing.  - Ability to contribute to designing and managing data warehousing solutions that optimize data storage and retrieval.  - Skills in data modeling, including entity-relationship diagrams, dimensional modeling, and semantic modeling.  - Track record of creating engaging data visualizations and interactive dashboards.  - Ability to perform data analytics, including exploratory data analysis and leveraging data for actionable insights.  - Some experience with implementing data governance frameworks and policies, including an understanding of managing data assets.  - Understanding of data security practices, including encryption and access control.  - Some knowledge of networking as it applies to configuring Azure data solutions.  - Understanding of DevOps or DataOps concepts and experience with the deployment of Azure data solutions.  - Proficiency in SQL and bonus for familiarity with any of the programming languages utilized within the Microsoft Data Platform stack, such as Python, Spark, C#, Java, Scala, R, DAX, MDX, or M.  - Advanced certification is preferred (see Recommended Qualifications).  - Bonus: Experience with Data Science and/or Machine Learning.    Competency with at least 3 and exposure to most of these technologies:  - Microsoft Fabric   - Azure Storage  - Azure Databricks (PySpark, SparkSQL, etc.)  - Power BI  - Git-based version control and deployments for Azure Data Solutions  - Microsoft Purview  - Bonus: Azure AI tooling  Your Educational and Professional Qualifications - Bachelor’s degree in a relevant discipline (e.g., MIS, CIS, Analytics) - 7+ years working in in professional services or the related industry. - Proficiency in Microsoft Office suite applications with specific emphasis on Word, Excel and PowerPoint. Secondary emphasis on Visio and Access. - Professional certification preferred: Microsoft Certified: Fabric Analytics Engineering Associate (highly desired), Microsoft Certified: Fabric Data Engineer Associate, Microsoft Certified: Azure Developer Associate, Microsoft Certified: Azure Administrator Associate, Microsoft Certified: Azure Solutions Architect Expert  Our Hybrid Workplace Protiviti practices a hybrid model, which is a combination of working in person with a purpose and working remotely. This model creates meaningful experiences for our people and our clients while offering a flexible environment. The ratio of remote to in-person requirements vary by client, project, team, and other business factors. Our people work both in-person in local Protiviti offices and on client sites, which can include local or out-of-state travel based on our projects and client requests and commitments. #LI-Hybrid Protiviti is not registered to hire or employ personnel in the following states – West Virginia, Alaska. Starting salary is based on a full-time equivalent schedule. Placement in the range is dependent upon experience, skills and geographic work location. Below is the salary range for this job. $159,000.00 - $255,000.00 Our annual bonus plan provides eligible employees additional cash and/or discretionary stock compensation opportunities. Below is the bonus target opportunity for this job. 14% The total cash range is estimated from the sum of the base salary range plus the bonus target opportunity. Below is the estimated total cash range for this job. $181,260.00 - $290,700.00 Employees are eligible for medical, dental, and vision coverages, FSA and HSA healthcare accounts, life and accident insurance, adoption and fertility assistance, paid parental leave up to 10 weeks, and short/long term disability. We offer eligible employees a company 401(k) savings and investment plan with an employer match of 50% on the first 6% of your contributions. We provide Choice Time Off (CTO) for vacation, personal needs, and sick time. The amount of (CTO) varies based on years of service. New hires receive up to 20 days of CTO per calendar year. Protiviti also recognizes up to 11 paid holidays each calendar year. Any benefits outlined are part of our reward offerings for full-time employees in the U.S. Your Open Enrollment materials, insurance contracts, plan documents and Summary Plan Descriptions together comprise the official plan document which legally governs the administration of your benefit plans. Protiviti reserves the right to terminate or amend your benefit plans in any way and at any time.

Related Categories

Related Job Pages

More Engineering Manager Jobs

Elation Health logo

Engineering Manager

Elation Health

Based in San Francisco, California, Elation Health is a privately held, mission-driven healthcare technology company that is committed to developing products that strengthen the ph

• Lead and participate in code reviews and share knowledge with the team • Ship your first AI-powered feature improvement to production • Partner with Product and UX to scope and design at least one new capability • Get familiar with our stack (Python, MySQL, React, AWS) and AI integration patterns • You're leading a team of 3-5 highly-engaged engineers and collaborating to improve team practices • Your team is shipping significant features end-to-end, from design through deployment • You're helping shape technical direction for AI-native product experiences • You've built strong partnerships with your team's Product Manager and UX Designer • You're actively measuring the value delivered and improvements to the patient and physician experiences in Elation • You're engaged as a player-coach and hands on shipping features with your team

Canada
$150K - $190K / year
ablefy logo

Engineering Manager – Platform & Infrastructure

ablefy

Empowering entrepreneurs to launch online courses, digital products, membership sites, and apps with our SaaS platform.

Full TimeRemoteTeam 51-200H1B No Sponsor

• You manage and develop engineers across DevOps, Platform, and AQA. • You will drive the decision process and execution to consolidate these into a coherent, maintainable stack — improving signal quality, reducing tool sprawl, and cutting cost. • You own the AWS infrastructure design for multi-region deployments — networking, data residency, compliance implications, and operational readiness. • Your Platform team works directly with stream-aligned teams to identify and resolve systemic performance risks — including query tagging, timeout analysis, and database-level bottlenecks that can cascade into incidents. • You own our CircleCI pipelines — speed, reliability, and cost. • Your AQA team owns automated testing across the organisation — Playwright-based end-to-end tests, test suite maintenance, and the quality gate that governs every release. • You work with your Platform team to improve the internal tooling that stream-aligned engineers depend on daily.

Germany
Job Closed
Full TimeRemoteTeam 501-1,000

At LightBox, we strive to not only equip confident, data-driven decisions across sectors, but to also enrich lives by bringing people, information, and technology together. As a company with a wide range of clients, we believe a diverse workforce is crucial to success. Our commitment to inclusion across race, gender, age, religion, identity, and experience is the foundation upon which we operate and connect with our customers and the communities in which we work. With our expertise, we are producing the best available data, workflow tools, technology, and analytics to support everyone making a real estate decision. There has never been a better time to make an impact and we invite you to join us on this journey. LightBox is a leading provider of data and workflow solutions across commercial real estate and location intelligence. Our solutions deliver the depth, speed and accuracy that enable insights to over 50,000 brokers, 1,000 banks and lenders, 1,000 law firms and 5,000 environmental consulting and engineering firms. The Role As Senior Engineer - AI Platform, you will build the self-service infrastructure and developer-facing tooling that makes LightBox's AI capabilities accessible and reliable. You'll work across the stack - from MCP server implementations and API gateways to internal developer tools and integration surfaces - ensuring that AI models and agents can be composed, deployed, and consumed by both internal teams and external partners. This role sits at the intersection of platform engineering and applied AI. You won't just be writing CRUD endpoints; you'll be building the connective tissue between proprietary CRE data, specialized AI models, and the production applications that hundreds of thousands of industry professionals use daily. What You'll Do - Design and implement API gateways, model routing layers, and service infrastructure that connect fine-tuned CRE models with production applications. - Develop internal developer tooling - prompt management interfaces, evaluation dashboards, trace viewers - that accelerate the team's ability to ship and iterate on AI features. - Implement integration patterns for embedding AI capabilities into LightBox's existing workflow applications, ensuring seamless user experiences. - Build observability and monitoring infrastructure for AI workloads: latency tracking, quality metrics, cost attribution, and usage analytics. - Contribute to the platform's knowledge retrieval and caching layers, optimizing for both accuracy and performance at scale. - Collaborate with ML engineers on model serving infrastructure, helping bridge the gap between training outputs and production deployment. - Participate in architecture decisions and technical design reviews, bringing a pragmatic full-stack perspective to the team's work. What We're Looking For Required - 6+ years of software engineering experience with a strong full-stack foundation (backend-leaning preferred). - Production experience building APIs, microservices, and platform infrastructure in Python and/or TypeScript/Node.js. - Solid understanding of cloud infrastructure (AWS or GCP), containerization (Docker, Kubernetes), and CI/CD pipelines. - Experience building developer-facing tools, SDKs, or internal platforms that other engineers consume. - Familiarity with LLM integration patterns: API-based model consumption, prompt engineering, retrieval-augmented generation, and tool/function calling. - Strong fundamentals in distributed systems, data modeling, and API design. - Comfort working in a small team with high autonomy - you can own a feature end-to-end from design through deployment and monitoring. Preferred - Experience with Model Context Protocol (MCP) or similar structured AI integration standards. - Background in data-intensive platforms, particularly property data, geospatial systems, or document processing pipelines. - Familiarity with model serving frameworks (LiteLLM, OpenRoute, AWS BedRock) and inference optimization. - Experience in real estate technology, financial services, or other regulated/document-heavy industries. - Prior work building observability or evaluation tooling for ML/AI systems. We thank all applicants in advance for their interest in this position, however, only those selected for an interview will be contacted. This job description is a general listing of the required tasks and expectations of the position and in no way implies that the duties listed above are the employee’s only responsibilities. The employee is expected to perform other tasks, responsibilities and training as instructed by their supervisors. Duties and responsibilities may change at any time with or without notice. This position may require additional hours outside of the standard work schedule including occasional holiday, evening and/or weekend hours in order to meet deadlines or to accommodate customers. LightBox and all its holding companies are an equal opportunity/affirmative action employer. It is the policy of the LightBox and its holding companies to prohibit discrimination of any type and to afford equal employment opportunities to employees and applicants, without regard to race, color, religion, sex, national origin, age, disability, or veteran status. NO TELEPHONE CALLS OR AGENCY SOLICITATION PLEASE.

United States
Full TimeRemoteTeam 501-1,000

At LightBox, we strive to not only equip confident, data-driven decisions across sectors, but to also enrich lives by bringing people, information, and technology together. As a company with a wide range of clients, we believe a diverse workforce is crucial to success. Our commitment to inclusion across race, gender, age, religion, identity, and experience is the foundation upon which we operate and connect with our customers and the communities in which we work. With our expertise, we are producing the best available data, workflow tools, technology, and analytics to support everyone making a real estate decision. There has never been a better time to make an impact and we invite you to join us on this journey. LightBox is a leading provider of data and workflow solutions across commercial real estate and location intelligence. Our solutions deliver the depth, speed and accuracy that enable insights to over 50,000 brokers, 1,000 banks and lenders, 1,000 law firms and 5,000 environmental consulting and engineering firms. The Role As Senior Engineer - AI Platform, you will build the self-service infrastructure and developer-facing tooling that makes LightBox's AI capabilities accessible and reliable. You'll work across the stack - from MCP server implementations and API gateways to internal developer tools and integration surfaces - ensuring that AI models and agents can be composed, deployed, and consumed by both internal teams and external partners. This role sits at the intersection of platform engineering and applied AI. You won't just be writing CRUD endpoints; you'll be building the connective tissue between proprietary CRE data, specialized AI models, and the production applications that hundreds of thousands of industry professionals use daily. What You'll Do - Design and implement API gateways, model routing layers, and service infrastructure that connect fine-tuned CRE models with production applications. - Develop internal developer tooling - prompt management interfaces, evaluation dashboards, trace viewers - that accelerate the team's ability to ship and iterate on AI features. - Implement integration patterns for embedding AI capabilities into LightBox's existing workflow applications, ensuring seamless user experiences. - Build observability and monitoring infrastructure for AI workloads: latency tracking, quality metrics, cost attribution, and usage analytics. - Contribute to the platform's knowledge retrieval and caching layers, optimizing for both accuracy and performance at scale. - Collaborate with ML engineers on model serving infrastructure, helping bridge the gap between training outputs and production deployment. - Participate in architecture decisions and technical design reviews, bringing a pragmatic full-stack perspective to the team's work. What We're Looking For Required - 6+ years of software engineering experience with a strong full-stack foundation (backend-leaning preferred). - Production experience building APIs, microservices, and platform infrastructure in Python and/or TypeScript/Node.js. - Solid understanding of cloud infrastructure (AWS or GCP), containerization (Docker, Kubernetes), and CI/CD pipelines. - Experience building developer-facing tools, SDKs, or internal platforms that other engineers consume. - Familiarity with LLM integration patterns: API-based model consumption, prompt engineering, retrieval-augmented generation, and tool/function calling. - Strong fundamentals in distributed systems, data modeling, and API design. - Comfort working in a small team with high autonomy - you can own a feature end-to-end from design through deployment and monitoring. Preferred - Experience with Model Context Protocol (MCP) or similar structured AI integration standards. - Background in data-intensive platforms, particularly property data, geospatial systems, or document processing pipelines. - Familiarity with model serving frameworks (LiteLLM, OpenRoute, AWS BedRock) and inference optimization. - Experience in real estate technology, financial services, or other regulated/document-heavy industries. - Prior work building observability or evaluation tooling for ML/AI systems. We thank all applicants in advance for their interest in this position, however, only those selected for an interview will be contacted. This job description is a general listing of the required tasks and expectations of the position and in no way implies that the duties listed above are the employee’s only responsibilities. The employee is expected to perform other tasks, responsibilities and training as instructed by their supervisors. Duties and responsibilities may change at any time with or without notice. This position may require additional hours outside of the standard work schedule including occasional holiday, evening and/or weekend hours in order to meet deadlines or to accommodate customers. LightBox and all its holding companies are an equal opportunity/affirmative action employer. It is the policy of the LightBox and its holding companies to prohibit discrimination of any type and to afford equal employment opportunities to employees and applicants, without regard to race, color, religion, sex, national origin, age, disability, or veteran status. NO TELEPHONE CALLS OR AGENCY SOLICITATION PLEASE.

United States