We are seeking a highly skilled and experienced Senior Data Engineer to join our client's AI technology team in London. In this role, you will be instrumental in designing, developing, and maintaining robust data pipelines and scalable data infrastructure to support complex AI and machine learning initiatives.
Key details:
Job Title: Senior/Lead Data Engineer
Contract: Permanent
Start date: ASAP
Salary: Up to £100,000 per annum + 0.4% equity
Location: Hybrid, 2 days on site a week in London
Key Responsibilities:
- Design, build, and maintain scalable and reliable data pipelines for real-time and batch processing.
- Develop and manage data architecture to support advanced analytics and AI models.
- Develop data modelling standards and ensure adherence across teams.
- Collaborate with cross-functional teams to understand data requirements and ensure efficient data integration.
- Optimise data workflows and storage for performance and cost-efficiency.
- Ensure data quality, consistency, and governance through best practices and tooling.
- Monitor and troubleshoot data pipelines and ensure system health and performance.
- Stay current with emerging data technologies, tools, and methodologies in the AI domain.
Required Skills and Experience:
- Proven experience as a Data Engineer, including work on production-grade systems.
- Strong programming skills in Python
- Extensive experience with distributed data processing frameworks such as Apache Spark, Flink, or Kafka.
- Proficiency in building and managing data pipelines using ETL/ELT tools.
- Experience with cloud platforms (AWS, Azure, or GCP) and associated data services.
- Solid understanding of data warehousing concepts and experience with tools like Snowflake, BigQuery, or Redshift.
- Hands-on experience with orchestration tools such as Airflow or Prefect.
- Familiarity with containerisation and CI/CD pipelines is desirable.
- Excellent communication skills and a collaborative approach to problem-solving.
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related discipline.
Interviews are taking place next week, so if you're interested, please apply today and we’ll be in touch shortly.
Alternatively, feel free to reach out to Josh Wolstenholme directly on LinkedIn for a confidential conversation.
