Distributed Systems Software Engineer

Description

We are looking for a highly motivated Distributed Systems Software Engineer to play a pivotal role in realizing Voltron Data’s streaming data vision within the composable data ecosystem. As a critical team member, you will be entrusted with meticulous component designs and driving executions to completion. You will collaborate alongside highly competent product engineering experts. You will also have the opportunity to work across open-source stacks (e.g., Ibis, Arrow, Substrait) and proprietary implementations.

Your contributions will directly impact the advancement of our unified data processing capabilities, drive innovation, and set new industry benchmarks. Join us in shaping the future of data analytics.

Why work at Voltron Data?

• We are Going for Impact: We are a Series A, venture-backed startup assembling a global team to design next-generation data systems, creating a new foundation for data processing built on composable open standards, with Theseus, our GPU query engine for petabyte-scale ETL, harnessing the speed and efficiency of modern hardware.

• We are Committed to Bridging Open Source Communities: We are a collection of open source maintainers who have been driving open source ecosystems over the last 15 years, particularly in the C++, Python, and R programming ecosystems.

• We are Building a Diverse, Inclusive Company:  We are creating a representative, equitable, and respectful workplace that prioritizes employee growth. Everyone at Voltron Data is bought into the company’s success; all voices are critical to shaping the organization’s future.

Timeline:

Below is a rough timeline of where you can expect to be at different points during your career path starting in this position.

Upon joining:

• Spending time learning about the Ibis and Apache Arrow, the open-source software we support as critical building blocks in the composable data ecosystem.

• Understand stream processing technologies that power data platform concepts such as Data Movement, Data Mesh, etc.

• Getting exposed to accelerated computing concepts leveraging hardware such as GPUs and DPUs.

• Learning and embracing the software development culture at Voltron Data.

Within a month:

• Engage actively in owning components from design to implementation in a native language like C++, Rust, or similar.

• Contribute to technical discussions and technical design documents.

• Work closely with open-source tools such as Ibis, Arrow, Kafka, DuckDb, Flink, and RAPIDS/cuDF.

• Contribute to bug-fixing efforts and propose areas for improvement in conjunction with the team.

Within 6 months:

Developing a comprehensive set of distributed stateless streaming operators and Arrow-based networking protocols to address the industry-wide data movement bottleneck. You will work with the team to implement an end-to-end system that significantly beats the current cost-efficiency benchmark.

• Leveraging columnar memory format, GPU compression, and stream processing techniques to transform an existing proof of concept into a more crafted offering.

• Identifying and building reusable components across the accelerated data movement code base.

Within 12 months:

Continuously profiling and analyzing throughput in a distributed system to identify inefficiencies,  and designing solutions to solve them. Incrementally evolving the streaming operators to support more complex stateful use cases in a unified compute engine.

• Mature the technology to be ready for broader adoption.

• Become one of the company's streaming go-to experts.

Previous experience that could be helpful (not all are required):

• You have built large-scale distributed systems or networking protocols that handle throughput greater than 10-100GB/s.

• You have worked with batch processing and stream processing systems. You are opinionated about data platform evolution and a strong thought leader.

• You have good intuitions about distributed system challenges and failure characteristics.

• You have experience planning capacity and estimations in cloud-native or on-premise data centers.

• You love building at a scale that most companies don’t even imagine.

• You have experience operating large-scale production systems and supporting your customers.

• You have experience with hardware like GPUs, DPUs, FPGAs, and associated software.

US Compensation - The salary range for this role is between $160,000 - $220,000. We have a global market-based pay structure which varies by location. Please note that the base pay range is a guideline, and for candidates who receive an offer, the exact base pay will vary based on factors such as actual work location, skills and experience of the candidate. This position is also eligible for additional incentives such as equity awards.

#LISM1

Benefits

• Work from Anywhere - Payroll and Benefits in 150+ Countries

• Unlimited PTO

• Medical, Dental, and Vision

• Retirement [USA Only]

• Home Office Budget

• Continuing Education Budget

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, sex, gender, gender expression, sexual orientation, age, marital status, veteran status, or disability status. We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.

To All Agencies: Please, no phone calls or emails to any employee of Voltron Data outside of the Talent Acquisition team. Voltron Data's policy is to only accept resumes from agencies via the Voltron Data Agency Portal. Agencies must have a valid fee agreement in place and they must have been assigned the specific requisition to which they submit resumes, by the Talent Acquisition team. Any resume submitted outside of this process will be deemed the sole property of Voltron Data and in the event a candidate is submitted outside of this policy is hired, no fee or payment of any kind will be paid