About the Role
You will be a technical leader responsible for the architecture, scalability, and reliability of a high-throughput, real-time data ecosystem. You will oversee the evolution of the data infrastructure, leveraging Kafka, RabbitMQ, Airflow, and ClickHouse to power mission-critical financial analytics. You will bridge the gap between complex business requirements and high-performance engineering, ensuring data pipelines can handle the rigours of real-time financial data processing.
Requirements
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field
- Proven experience in data engineering, with a strong background in designing and implementing ETL processes within cloud environments
- Experience within the Finance or Trading technology sector, with a proven track record of handling real-time market or transactional data
- Strong programming skills in Python, with experience in developing robust, maintainable, and scalable data processing pipelines
- Extensive SQL knowledge and experience
- Excellent problem-solving skills and the ability to work collaboratively in a team environment
- Strong communication skills, with the ability to convey complex technical concepts to non-technical stakeholders
Responsibilities
- Lead the design and evolution of highly scalable, fault-tolerant ETL/ELT pipelines
- Drive the strategy for real-time messaging and stream processing using Kafka and RabbitMQ to ensure sub-second data availability
- Act as the subject matter expert for ClickHouse, optimising complex schema designs, indexing strategies, and query performance for large-scale financial datasets
- Oversee the deployment of data services within cloud environments, implementing advanced security protocols and data governance standards
- Collaborate with senior leadership to align data strategy with business objectives
- Mentor data engineers through code reviews and technical guidance
- Implement advanced monitoring and automated recovery systems to ensure the integrity and quality of high-stakes financial data
Benefits
- Hybrid working arrangement - 2 Days of remote work per week
- Complimentary snacks and beverages available in the office pantry
- Healthcare coverage (medical, dental, optical), gym benefits
- Flexibility in smart casual dress code