Building Advance AI & Cloud Native Software Using The "Virtual Silicon Valley" Model. Let’s Talk AI, Cloud and Outcomes.
Fullstack Data Engineer
Location
United States
Posted
93 days ago
Salary
0
Seniority
Lead
Job Description
Fullstack Data Engineer
Codvo.ai
• Design, build, and maintain Databricks data pipelines (ETL/ELT) for ingestion, transformation, and orchestration using Spark/Delta Lake/Databricks Workflows. • Operationalize machine learning models by building inference pipelines that invoke models authored by data scientists (batch or real-time), ensuring consistency between training and inference environments. • Ensure data reliability, quality, and observability through robust validation, monitoring, alerting, and automated recovery mechanisms. • Collaborate closely with data scientists to productionize models, manage model deployment lifecycles, and optimize inference performance and cost. • Implement best-practice DevOps/MLOps processes such as CI/CD for pipelines, model versioning, environment promotion, and infrastructure-as-code. • Optimize performance and cost across compute clusters, jobs, and storage layers. • Implement and manage the enterprise data catalog, including schema design, table ownership, lineage, governance, and documentation using Unity Catalog. • Experience with some Databricks infrastructure. • Experience with building BI dashboards and visualization. • Experience with coding agents and best practices (spec-driven development, etc.)
Job Requirements
- 8+ yrs experience
- Databricks platform experience
- Python development for data processing and ETL pipelines
- Unity Catalog knowledge
- AWS data services (S3, IAM, VPC, potentially Glue/Lambda)
- Data lake/lakehouse architecture patterns
- Dashboard building experience
- RESTful API design and development (Flask, FastAPI, or similar)
- Authentication/authorization patterns (OAuth, API keys, IAM roles)
- Query optimization and performance tuning
- PySpark optimization experience
- ML/AI pipeline experience
- Databricks AI/BI
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Data Architect
Creative Information Technology, Inc.Creative Information Technology Inc (CITI) is an esteemed IT enterprise renowned for its exceptional customer service and innovation. We serve both government and commercial sectors, offering a range of solutions such as: Healthcare IT Human Services Identity Credentialing Cloud Computing Big Data Analytics With clients in the US and abroad, we hold key contract vehicles including: GSA IT Schedule 70 NIH CIO-SP3 GSA Alliant DHS-Eagle II Join us in driving growth and seizing new business opportunities. Preferred Qualification Prior experience working with public health agencies (e.g., CDC, NIH, or State Departments of Health) to modernize legacy data infrastructures.
Role Description The scope of the proposed services will include the following: - Design the vision and blueprint of an organization’s data framework. - Design and manage data systems, set policies for how data is stored and accessed. - Coordinate various data sources within an organization. - Integrate new data technologies into existing IT infrastructures. - Data modeling and design, including software development and database administration. Qualifications - Ten (10) plus years professional experience in data ecosystems writing SQL queries, building ETL pipelines, and designing data architecture. - Bachelor’s degree in computer science, Computer Engineering, Data Science, Management Information Systems, or Information Technology. Company Description Creative Information Technology Inc (CITI) is an esteemed IT enterprise renowned for its exceptional customer service and innovation. We serve both government and commercial sectors, offering a range of solutions such as: - Healthcare IT - Human Services - Identity Credentialing - Cloud Computing - Big Data Analytics With clients in the US and abroad, we hold key contract vehicles including: - GSA IT Schedule 70 - NIH CIO-SP3 - GSA Alliant - DHS-Eagle II Join us in driving growth and seizing new business opportunities. Preferred Qualification - Prior experience working with public health agencies (e.g., CDC, NIH, or State Departments of Health) to modernize legacy data infrastructures.
Role Description We are looking for a well-rounded, flexible Data Engineer with an interest in translating business goals into concrete solutions. You’ve been part of more than one team, you’ve demonstrated initiative on most of your projects, and you can tell us in technical detail what you did to help. Ideally you’ve worked with at least one devops team, and in at least one flavor of agile. If you haven’t we won’t hold that against you. At DEPT®, we pride ourselves on our pragmatic, engineering approach to making data work for the business. Qualifications - Expertise across multiple technical and functional domains to identify and execute optimal technical solutions. - Ability to understand design decisions and articulate the tradeoffs made. - Aware of industry trends and how they affect our work, introducing new ideas into current workflows. - Offer advice and opinions for technical decisions affecting multiple clients/projects or the entire company. - Communicate context, recommendations, and rationale to relevant internal and/or external stakeholders. - Demonstrate the ability to question and ensure that work maximizes client value delivery. - Synthesize input from various team members to drive to a solution. Requirements - 7+ years of experience in several of the following: - Pipeline management tools like Airflow or Luigi, and/or enterprise ETL tools like Informatica, SSIS, or Datastage. - Scripting languages like Python, Ruby. - Compiled languages like Java, Scala, C++. - Database Management Systems like Databricks, BigQuery, Snowflake, PostgreSQL, mySQL, Oracle, MS SQL Server, Redshift, MongoDB. - Experience with Scala, Kafka, and MongoDB is a plus. - Experience working in cloud environments (AWS or GCP) is a plus. - Experience with Unix-based OSes is a plus. - 5+ years of discovery and assessment of a client’s portfolio, tools, and applications. - 3+ years of professional software and consulting experience. - Experience managing smaller project teams. - Experience with data governance, observability, and quality frameworks. - Ability to take a project from RFP/SOW/BRD to delivery: project management, requirements gathering, data architecture, workflow management, quality assurance, reporting and visualization, presentation. - Interest in mentoring junior colleagues on process, analysis, deliverables, client. Benefits - Healthcare, Dental, and Vision coverage. - 401k plan, plus matching. - PTO. - Paid Company Holidays. - Parental Leave. Company Description DEPT® is a Growth Invention company built to help the world’s most ambitious brands grow faster. Operating at the intersection of technology and marketing, our 4,000+ specialists deliver growth invention across Brand & Media, Experience, Commerce, CRM, and Technology & Data. We’re 50|50 tech and marketing, partner-led, and first to move. Clients include Google, Lufthansa, Coach, eBay, and OpenAI.
• Data Collection: Collect data from various internal and external sources, following established procedures and proposing process improvements when necessary. • Data Preparation: Perform data cleaning, transformation, and integration tasks, ensuring data quality and consistency with minimal supervision. • Project Support: Collaborate with team members on development and maintenance of data pipelines, taking on larger responsibilities and leading small subprojects. • Process Documentation: Contribute significantly to documentation of data engineering processes, recording procedures and workflows and ensuring they stay up to date. • Exploratory Data Analysis: Conduct initial data analyses to identify patterns, trends, and business-relevant insights with greater autonomy. • Query Support: Execute more complex SQL queries and extract information from relational databases, providing technical support to the team. • Quality Monitoring: Proactively monitor data quality, identify and resolve issues or inconsistencies, and suggest improvements to quality control practices. • Continuous Learning: Participate in and contribute to training and development programs, as well as mentor junior team members. • Cross-functional Collaboration: Work effectively with other departments to ensure proper integration of data into solutions and systems, contributing specialized knowledge. • Meeting Deadlines: Manage your tasks and ensure they are completed within established deadlines, proactively communicating any challenges or obstacles to team leadership.
Visual Data Collector (Freelance)
RAY AISelection is rigorous. Standards are high. If that sounds right for you -> apply.
Project: Real-World Image Capture for AI Model Training Location: Remote / Field-Based Engagement Type: Freelance, On-Demand (Client-Driven Volume) Compensation: Pay-Per-Series About the Role We are looking for reliable, detail-oriented individuals to join our global network of Visual Data Collectors supporting AI model training. This role does not require professional photography experience. Instead, success in this position depends on your ability to follow detailed instructions precisely and capture authentic, real-world moments that reflect everyday life. Your contributions will help train AI systems to better understand lighting, perspective, depth, and environmental context using real, unedited imagery. How the Work Operates - On-Demand Projects: Work availability is based on client demand. Some periods may have higher volume than others. - Flexible Schedule: When projects are active, you work on your own schedule, provided deadlines are met. - Output-Based: Compensation is strictly tied to approved deliverables, not time spent. Scope of Work You will create photographic Series, each representing one real-world scenario. What is a Series? - One scenario captured through 5–6 required photos - Examples include: someone drinking coffee, fixing a light, walking in a park, or a family picnic - Images must look natural and unstaged Important Notes - Photos do not need to be artistic or complex - Accuracy, clarity, and realism are essential - All images must strictly follow provided technical and content guidelines Key Responsibilities - Capture authentic, believable real-life scenarios - Ensure correct focus, lighting, and environmental consistency - Organize and label images according to required metadata standards - Submit completed series via Google Drive - Obtain signed indemnity forms for all human subjects (template provided) - Avoid visible logos, brand names, or copyrighted materials Requirements - Equipment: Smartphone or digital camera capable of at least 7 MP images - Reliability: Ability to follow instructions exactly—near-correct data cannot be used - No Editing: Images must be raw; no filters, retouching, or AI tools permitted - Access: Ability to shoot in varied environments (home, streets, parks, cafés, etc.) - Compliance: Willingness to adhere strictly to ethical and legal data-collection standards Compensation & Payment Terms - Base Rate: $3.50 per approved Series (each Series = 5–6 photos of one scenario) - Pay-Per-Output: Payment is issued only for submissions that fully meet quality and guideline requirements - Efficiency Potential: Contributors who optimize their workflow may earn the equivalent of up to $28/hour based on approved output (not guaranteed) - Volume Incentives: Higher rates or bonuses may be offered to contributors who consistently submit 100+ approved series - No Fees: We do not charge applicants or contributors any fees at any stage Next Steps If you have: - A smartphone or camera - Access to everyday environments - Strong attention to detail - The ability to follow detailed instructions We encourage you to apply. Selected applicants will receive full documentation outlining file specifications, metadata requirements, and submission workflows.



