Transformamos talentos em resultados!
Senior KDB+/Q Developer, Time Series Data Engineer
Location
Brazil
Posted
56 days ago
Salary
0
Seniority
Senior
Job Description
Senior KDB+/Q Developer, Time Series Data Engineer
talent's club
• Q Development: Write efficient and concise code in the Q language for manipulating in-memory and disk (splayed/partitioned) tables; • Fluent English; • Feed Architecture: Develop and optimize Ticker Plants, Chained Tickers, and Real-time Databases (RDB); • Historical Data Management: Maintain and optimize the Historical Database (HDB), ensuring that queries on terabytes of data return in milliseconds; • Performance Optimization: Identify I/O and CPU bottlenecks, applying vector processing techniques to avoid slow loops and iterators; • Integration: Create connectivity interfaces (APIs) between kdb+ and other languages such as Python (PyQ/EmbedPy), Java, or C++.
Job Requirements
- kdb+ Proficiency: Solid experience with the database structure and the q language.
- Vector Thinking: Proven ability to solve complex problems using native functions (adverbs, over, scan) instead of procedural logic.
- Operating Systems: In-depth knowledge of Linux/Unix (shell scripting, memory management, and file systems).
- Low Latency Architecture: Understanding of how kdb+ interacts with hardware (RAM vs SSD) for query optimization.
- Problem-solving mindset under pressure (real-time production environments).
- Knowledge of messaging protocols (Solace, Kafka) and frameworks such as Aqua or TorQ (Desirable).
- Previous experience in Financial Markets (Equities, FX, Fixed Income) or analysis of large volumes of sensors (IoT) (Desirable).
Benefits
- Health insurance
- Flexible working hours
- Professional development opportunities
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Full‑Stack data engineer We are seeking a highly self‑sufficient, motivated engineer based with strong full‑stack data engineering skills to join our innovative, dynamic team. This is a remote/offshore role that requires autonomy, excellent communication, and the ability to deliver high‑quality work with limited supervision while collaborating with a predominantly US‑based team. You will build reliable, scalable data products and user experiences that power AI/ML modeling, agentic workflows, and reporting—working end‑to‑end from data ingestion and transformation through to UI. Our Python‑based data platform is undergoing a major evolution toward a modern, cloud‑native ELT architecture. We are standardizing on Snowflake as our central data platform and dbt as our core transformation framework, implementing scalable, maintainable ELT practices that simplify ingestion, modeling, and deployment. This role will be pivotal in independently designing and building robust data pipelines and semantic layers that directly power our AI and machine learning initiatives—delivering clean, reliable, and well‑modeled data assets to our data science team for feature engineering, model training, and production inference. You will collaborate closely (primarily via remote channels) with data scientists and ML engineers to ensure our data ecosystem is optimized for experimentation speed, model performance, and seamless integration into downstream products and services. Key Responsibilities - Remote collaboration & communication: Operate effectively as an offshore member of a distributed team, proactively communicating status, risks, and blockers across time zones and coordinating overlap with US working hours as needed. - Full‑stack data engineering: Build across the entire stack, including data ingestion/acquisition and transformation, APIs, front‑end components, and automated test suites, delivering production‑grade solutions with minimal hand‑holding. - Autonomous delivery & ownership: Take end‑to‑end ownership of features and projects—clarifying requirements, breaking work into milestones, estimating timelines, and delivering high‑quality, well‑documented solutions. - Specification and design: Translate short‑ and long‑term business requirements, architectural considerations, and competing timelines into clear, actionable technical specifications and design documents. - Code quality: Write clean, maintainable, efficient code that adheres to evolving standards and quality processes, including unit tests and isolated integration tests in containerized environments. - Continuous improvement: Contribute to agile practices and provide input on technical strategy, architectural decisions, and process improvements, continuously suggesting better tools, patterns, and automation. Required Skills & Experience - Professional experience: 5+ years in software engineering, with a full‑stack background building complex, scalable data‑engineering pipelines using data warehouse technology, SQL with dbt, Python, AWS with Terraform, and modern UI technologies. - Modern data engineering: Strong experience with medallion data architecture patterns using data warehouse technologies (e.g., Snowflake), data transformation tooling (e.g., dbt), BI tooling, and NoSQL data marts (e.g., Elasticsearch/OpenSearch). - Testing and QA: Solid understanding of unit testing, CI/CD automation, and quality assurance processes for both data pipeline testing and operational data quality tests. - Remote work & autonomy: Proven track record working in a remote or distributed environment, demonstrating self‑motivation, reliable execution, and the ability to make sound technical decisions independently. - Agile methodology: Working knowledge of Agile development practices and workflows (e.g., sprint planning, stand‑ups, retrospectives) in a distributed team setting. - Education: Bachelor’s or Master’s degree in Computer Science, Statistics, Informatics, Information Systems, or a related quantitative field. Preferred Skills & Experience - Machine learning and AI: Hands‑on experience with large language models (LLMs) and agentic frameworks/workflows. - Search and analytics: Familiarity with the ELK stack (Elasticsearch, Logstash, Kibana) for search and analytics solutions. - Cloud expertise: Experience with AWS cloud services; familiarity with SageMaker; and CI/CD tooling such as GitHub Actions or Jenkins. - Front‑end expertise: Experience building user interfaces with Angular or a modern UI stack. - Financial domain knowledge: Broad understanding of equities, fixed income, derivatives, futures, FX, and other financial instruments.
Software Engineer II - Azure Data Lake Storage Gen 2
MicrosoftMicrosoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to any characteristic protected by applicable local laws, regulations, and ordinances.
Overview Are you excited about the possibility of working on the largest Cloud Storage service in the world? Azure Storage team is chartered with building, managing and running the Persistent Cloud Storage for Microsoft Azure Cloud. Azure Storage consists of a Storage Platform which currently powers five different storage services: Blob Storage, Table Storage (NoSQL Key Value store), Queue Storage, Data Lake Storage, File Storage and Disk Storage. We are one of the foundational services in the Azure Cloud. The service hosts data from some of the largest companies in the world plus supports all of Microsoft’s largest online businesses including Xbox, Teams and Bing and has scaled to store exabytes of customer data with the roadmap looking even better in terms of growth. We are looking for a Software Engineer II - Azure Storage with strong coding and problem solving skills and a solid foundation in designing and building reliable software. This position will contribute to building and improving key components of Azure Storage, delivering customer and partner facing features and reliability improvements with high quality. The candidate will work on modern server technologies and highly distributed systems, with a focus on reliability, scalability, performance, consistency and debuggability. If you are ready to take on the challenge of solving impactful problems, designing and implementing well scoped components, and debugging live systems in partnership with experienced engineers, then this is the position for you. Strong collaboration skills, the ability to communicate clearly with teammates and partners, and a strong commitment to our customers will be required. Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. Responsibilities - Collaborates with appropriate stakeholders to determine user requirements for a scenario, asks clarifying questions, and helps translate requirements into engineering tasks. - Contributes to the development of design documents for a product, application, service, or platform by proposing options, identifying tradeoffs, and reviewing designs with senior engineers. - Creates, implements, optimizes, debugs, refactors, and reuses code to improve correctness, performance, and maintainability. - Writes and maintains unit, integration, and scenario tests, and uses telemetry and diagnostics to validate health and quality in production. - Delivers well scoped work items end to end, including implementation, code reviews, testing, safe rollout, and follow up based on signals and feedback. - Partners with appropriate stakeholders (for example, project managers and partner teams) to execute on project plans, release plans, and work items for owned areas. - Participates in on call rotation and live site operations by following established playbooks, escalating appropriately, and contributing to mitigation and post incident follow up. Qualifications Required Qualifications: - Bachelor's Degree in Computer Science or related technical field AND 2+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python - OR equivalent experience. Other Requirements: - Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: - Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter. Preferred Qualifications: - Master's Degree in Computer Science or related technical field AND 3+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python - OR Bachelor's Degree in Computer Science or related technical field AND 5+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python - OR equivalent experience. #azurecorejobs This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled. Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
WHAT YOU'LL BE WORKING ON - Set up a data migration process from one database to another - Assist with automation of BI processes by building new ETL and ELT scripts - Support and improve current Analytics infrastructure (incl. Airflow) - Review scheduled jobs and queries, perform costs optimization - Work with different partners API sources and implement data upload - Improve analytics processes, set up cross checks and data lineage Our Tech Stack: DB - BigQuery; Language - Python; BI - Tableau WHAT YOU NEED TO SUCCEED IN THIS ROLE - Databases: experience with transactional and analytics databases (Google BigQuery, Snowflake), advanced SQL knowledge - API: working with different APIs and setting up data loads - ETL/ELT Data Pipelines: build and support piplines with Python - Python: data Analysis and preprocess with Python - Airflow: DAGs creation, sensors, Airflow infrustructure support - Additional (would be a benefit): Tableau experience, working with clouds (AWS), Data Science skills - Strong analytical and problem-solving skills - Bachelor's degree in a related field such as Statistics, Mathematics, Computer Science, or a similar quantitative discipline would be a plus - Good level of English and russian language. WHAT WE OFFER - An opportunity to make something great even greater, you can be the reason why we grow, develop, and become the best fintech company on the market! - Career prospects - we are young, we have huge ambitions, and it is important that our employees grow with us - Work with coworkers who are passionate about their business - Compensation that will fully correspond to the competence and knowledge, with yearly performance reviews - The opportunity to attend conferences, courses, and professional development at the company's expense - we favor the continuous development of our employees - Remote type of work - 20 days of vacation time; Latvian bank holidays; Sick leaves; Additional birthday day off You can find out more about our team, our values, and the unique ways we celebrate our successes here: FYST Culture
WHAT YOU'LL BE WORKING ON - Set up a data migration process from one database to another - Assist with automation of BI processes by building new ETL and ELT scripts - Support and improve current Analytics infrastructure (incl. Airflow) - Review scheduled jobs and queries, perform costs optimization - Work with different partners API sources and implement data upload - Improve analytics processes, set up cross checks and data lineage Our Tech Stack: DB - BigQuery; Language - Python; BI - Tableau WHAT YOU NEED TO SUCCEED IN THIS ROLE - Databases: experience with transactional and analytics databases (Google BigQuery, Snowflake), advanced SQL knowledge - API: working with different APIs and setting up data loads - ETL/ELT Data Pipelines: build and support piplines with Python - Python: data Analysis and preprocess with Python - Airflow: DAGs creation, sensors, Airflow infrustructure support - Additional (would be a benefit): Tableau experience, working with clouds (AWS), Data Science skills - Strong analytical and problem-solving skills - Bachelor's degree in a related field such as Statistics, Mathematics, Computer Science, or a similar quantitative discipline would be a plus - Good level of English and russian language. WHAT WE OFFER - An opportunity to make something great even greater, you can be the reason why we grow, develop, and become the best fintech company on the market! - Career prospects - we are young, we have huge ambitions, and it is important that our employees grow with us - Work with coworkers who are passionate about their business - Compensation that will fully correspond to the competence and knowledge, with yearly performance reviews - The opportunity to attend conferences, courses, and professional development at the company's expense - we favor the continuous development of our employees - Remote type of work - 20 days of vacation time; Latvian bank holidays; Sick leaves; Additional birthday day off You can find out more about our team, our values, and the unique ways we celebrate our successes here: FYST Culture


