About the Role
One of Sibros’ core products is to offer OTA (over-the-air) data collection to a fleet of vehicles. We believe all the new vehicles will have some form of connectivity to the cloud in a few years. We are building large-scale data warehouse solutions and real-time data processing pipelines to handle billions of messages coming from millions of vehicles. Our platform has demanding requirements in latency, efficiency and scalability. As a part of the team, you will be working on leveraging various open source technologies to solve problems at the planet scale in a high read throughput and high write throughput environment. The platform is/will be handling terabytes to petabytes of data, and ingesting billions of rows per day. You will also be working on how to transform the data into valuable information for the end-users. Your responsibilities include, but are not limited to:
- Design and implement scalable data pipelines to process millions of records per second, and ingest terabytes of data into various of storages.
- Build a generic data pre-processing / post-processing system that transforms data coming from vehicles, while allowing complex, user-defined workflows
- Understand the limitation of various data warehouse solutions, and leverage them to build a comprehensive platform that satisfies speed, efficiency and scalability
- Optimize the solution based on query patterns and come up with a design that scales horizontally
- Collaborate closely with firmware engineers, frontend engineers and product owners to build a highly scalable data visualization platform
- Create easily to maintain data API for end-users to integrate with the system
- Excellent understanding of computer science fundamentals, data structures, and algorithms
- Strong track record in designing and implementing large scale distributed systems
- Willingness to wear multiple hats and adapt to a fast-paced startup environment
- Proficient in writing at least one programming language in production-grade
- Passionate about the vision and mission of the company, and interested in solving challenging problems in the automotive IoT domain
- Strong experience in writing Golang in production
- Experience in designing and implementing systems for large scale IoT devices
- Experience in the public cloud and cloud-native technologies. E.g. AWS, GCP, Azure.
- Extensive knowledge of pros and cons of popular storage systems
- Extensive knowledge in various data lake solutions like Athena, BigQuery.
- Experience in working with large scale time-series database such as Apache Druid, Pinot.
Equal Employment Opportunity
Sibros is committed to a policy of equal employment opportunity. We recruit, employ, train, compensate, and promote without regard to race, color, age, sex, ancestry, marital status, religion, national origin, disability, sexual orientation, veteran status, present or past history of mental disability, genetic information or any other classification protected by state or federal law.