Enabling smarter investment decisions and better client communications.
Software Engineer, Data
Location
New York
Posted
3 days ago
Salary
$120K - $160K / year
Seniority
Mid Level
Job Description
Software Engineer, Data
YCharts
• Work with technologies that ensure our data pipeline efficiently maintains and calculates data for the 7 million+ securities in our universe. • Work on features that ensure financial data is consistent, reliable, timely and accurate by importing data from third party data providers and performing custom calculations on this imported data, with day-to-day emphasis on database load management, caching strategies, memory efficiency, and query optimization. • Write code (and unit tests!) in Python and the Django framework to implement complex backend features. • Use tools like Airflow to build / enhance financial data pipelines for over 7 million securities. • Make use of pandas and asynchronous task tools like celery to efficiently calculate 4,000+ unique data points for each security. • Profile and optimize database queries — analyze query plans, design effective indexes, and eliminate N+1s and slow paths that don't hold up under production load. • Manage database load — reason about read/write patterns, connection usage, replica strategy, and batching to keep the data tier healthy as volume grows. • Design and tune caching layers to reduce redundant computation and database pressure while keeping data fresh and correct. • Design robust, scalable data models that account for the structure, type, and volume of data to store, and how it will be queried. • Investigate and resolve performance regressions and incidents — using metrics, profiling, and monitoring to find root causes. • Design, plan, estimate, and ticket features that are scoped by our product team. • Perform code reviews for other engineers on the team. • Document the work you have done both in repo as well as outside of the repo for future engineers. • Collaborate with other engineers both verbally and in writing to plan, design, and build a world-class financial research platform.
Job Requirements
- 2+ years of relevant industry or academic experience
- Experience developing in Python (knowledge of the Django framework a plus)
- Experience designing, building and maintaining application features on the backend
- Experience working with relational databases, including writing and optimizing queries and reasoning about indexing and query performance
- Experience participating in multi-month projects from conception to maintenance with multiple team members
- Familiarity with caching, and strategies for managing database and memory load at scale a plus
- Experience working with large datasets and an interest in the performance and efficiency challenges that come with them
- Desire to take ownership and responsibility when a problem or opportunity arises
- Experience working with non-technical teams (product, business, etc) where explaining technical concepts is needed
- Self-organized and able to work independently.
Benefits
- 100% employer-paid health, dental, and vision insurance.
- 401(k) match to support your financial future.
- Opportunities for internal mobility and cross-functional collaboration.
- Flexible time off, vacation days, sick days, and a celebration day.
- Paid parental leave to support work-life balance.
- Professional development stipend to help you grow in your career.
- Hybrid work schedule with flexibility to work both in-office and remotely.
- Summer hours so you can enjoy more sunshine.
- Recognition programs to celebrate contributions and milestones.
- Modern, centrally located offices stocked with premium snacks, coffee, beverages and weekly lunch credits to fuel your day.
- Regular team events, celebrations, and company-wide gatherings that keep our culture connected and collaborative.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Partner with Enterprise Architects, Senior Architects, and cross functional teams, including business stakeholders, to shape solution architectures aligned to a unified IT data strategy. • Contribute to Empower’s technology roadmap by documenting target state architectures, reference architectures, and key dependencies for supported initiatives. • Design and support modernization and migration of complex systems to scalable, secure, and cost optimized cloud architectures, working with delivery teams through implementation. • Evaluate and prototype new technologies and applications, challenge existing approaches constructively, and provide input into business cases and recommendations. • Apply and help operationalize architectural standards, best practices, and governance expectations, supporting consistent adoption through guidance and enablement. • Collaborate closely with business and engineering teams to design end to end solutions that meet strategic, technical, and operational objectives. • Provide architectural guidance in process design, system orchestration, and enterprise integration to enable cohesive and efficient technology ecosystems. • Champion security, resilience, and scalability as core design principles across data platforms and data solutions. • Contribute to architecture discipline maturity by producing reusable patterns, decision documentation, and lessons learned that improve consistency and delivery speed. • Support full stack development teams through solution design and implementation, improving long term scalability, performance, and cost outcomes.
Staff Engineer, Market Data
Douro LabsDouro Labs is the company behind Pyth Pro, real-time market data infrastructure that distributes institutional-grade price feeds from primary sources directly to exchanges, market makers, and trading firms. Pyth Pro currently provides the market data behind many major cryptocurrency exchanges and prediction markets, with a growing base of institutional users relying on it daily.
• Build and maintain core market data infrastructure in Rust • Work through the practical problems specific to real-time market data: cross-exchange pricing differences, data quality, latency • Bring real market data or trading systems knowledge into engineering decisions • Communicate clearly with the rest of the team as the group grows • Take on leadership of a small team over time, not immediately
Role Description Experienced professional specializing in data segmentation, processing, and list management for direct marketing and merge purge campaigns. Skilled in leveraging CRM data, working with complex matching logic, and applying business rules to optimize performance. Adept at delivering high-quality, on-budget, and on-time results while building strong client relationships and ensuring exceptional service. Proven record of accomplishment of implementing process improvements and automation to enhance operational efficiency. Strong technical proficiency with data integration and cleansing tools, combined with the ability to collaborate effectively with both technical and non-technical stakeholders. This is a senior role that will require delivering large-scale process improvement efforts that span across the team and services. This full-time, salaried position is a US-based remote role. Your Impact - Accountable for maintaining a high degree of Client Satisfaction that includes internal team partners as "Clients". - Accurately records business metrics: Time Tracking, Quality, and Billing per established business and team policies. - Challenges the status quo for implementing process improvement at an individual level while also recommending improvements at the team level. - Develop strong relationships with both internal and external partners while balancing client satisfaction against strong service delivery. - Effectively communicates project status and risks in a timely fashion to leadership and account manager. - Ability and willingness to adopt and contribute to change while avoiding change for the sake of change. - Balances the needs of the business, clients, and team, applying business acumen to decision-making. - Ensures comprehensive, clear, and accurate documentation of assigned work. - Identifies risk of challenges and conflicts, escalating them to management for remediation. - Completes other job-related tasks as assigned. Your Profile - Previous experience in Non-Profit Donor Fundraising Acquisition Merge-Purge preferred. - A bachelor's degree in a related field or equivalent experience. - A strong professional background in segmentation, list pulls, data hygiene, and data processing services. - Experience with SQL and the ability to successfully work with disparate data across multiple sources using multiple tools: E.g., SSIS, SSMS, Alteryx. - An in-depth understanding of donor management, data services, data processing, and analytics. Your Skills - Exceptional verbal and written communication skills. - The ability to work cooperatively with your peers and management team. - A high level of organization and a keen eye for detail. - An ability to simultaneously manage multiple projects while demonstrating the competency to prioritize and meet timelines. - Experience working with structured and unstructured data. - An ability to troubleshoot errors and overcome technical challenges. - Provable experience in process improvement and quality control practices and methodologies. - Maintains self-awareness in interactions and fosters strong relationships, especially during conflicts. - Adept at working with changing landscapes and resisting stagnation due to the number of details or challenges. Joining Moore, You Will - Do meaningful, life-changing work every day by supporting the nation's most beloved nonprofits and service organizations. - Join the largest marketing and fundraising company in North America serving the nonprofit industry where we prioritize innovation and professional growth. - Collaborate with industry subject matter experts among over 5,000 employees across the enterprise. - Earn a competitive salary and have access to comprehensive health, wellness, and retirement benefits. - Enjoy paid holidays and generous paid time off, ensuring you have the time and space to recharge and pursue your other passions. Moore's Commitment to Candidates Moore is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. Moore is also committed to compliance with all fair employment practices regarding citizenship and immigration status.
Role Description Talend developer with experience designing and delivering complex, large-volume data warehouse applications. - Senior Level Talend ETL development (4+ years of hard-core Talend experience) - Teradata experience, including Multiload, Fastload, T-Pump and preferably TPT - Experience in Big Data and Snowflake is preferred - Preferably has exposure in Map Reduce components in Talend - Strong experience in advanced SQL programming - Minimum data modeling experience - High volume data warehouse design experience taking Teradata unique nuances into account such as skewing considerations - Solid and professional communications skills, both verbal and written - Unix familiarity and shell scripting experience - Strong knowledge of Software Development Lifecycle (SDLC) - Job scheduling experience using Maestro is required - Strong experience in Data Quality, Source Systems Analysis, Business Rules Validation, Source Target Mapping Design, Performance Tuning and High Volume Data Loads - Strong written and oral communication skills are essential - Strong analytical skills and ability to resolve problems are desired - Some knowledge of the mortgage industry required - Ability to work independently and multi-task to meet critical deadlines in a rapidly changing environment - Working knowledge of source-code control using any tool available in the market is preferred - High energy, self-motivation, attention to detail, creativity, flexibility, and ability to work under pressure - Flexibility to work additional hours on an as-needed basis to meet deadlines and with offshore team Requirements - Total IT experience of 6+ years Roles & Responsibilities - Analyze, design, implement and maintain high volume, multi-terabyte 24/7 data warehouse robust and fast ETL applications - Advanced ETL development using Talend Big Data platform - Be a senior technical player on complex ETL development projects with multiple team members - Develop relevant functional and technical documentation - Lead, design and develop data warehouse solutions using different architecture, design and modeling techniques


