Job Description:The Opportunity The Senior Data Engineer will join a team developing a world class data platform to support our customers, products, researchers, and back-office teams. You’ll join a small squad of Data Engineers and QA specialists empowered to define ways of working that work best for all. You’ll be adding features, developing data pipelines, automating the ETL of complex data and designing and configuring platform solutions that support a wide variety of applications. You’ll have the freedom to come up with innovative solutions to unique challenges and help us use data as a force for good. What we can offer you: You will design and implement the data storage model, defining the rules for data governance, data quality, and the data lifecycle. You will work closely with the Information Architect, Product Owners, and Data Platform stakeholders, ensuring high level designs. Be responsible for ingesting data into our data lake and provide frameworks and services for operating on that data. Design and support data pipelines and recommend improvements and modifications. Build the process required for optimal extraction, transformation, and loading of data from a wide variety of data sources. Develop and implement processes to ensure data accuracy and integrity. Collaborate closely with our cloud infrastructure team on shared items of work and communicate well. You will proactively ensure code reviews are prioritised to help unblock the team’s work. You will identify and help others to resolve blockers, aiding delivery. The skills you will bring: Designed and built scalable, data-centric systems with over 6 years of engineering experience, including 3 years of specialized focus on processing and transforming large-scale data for enterprise-level systems. You have hands-on experience with AWS services like Data Pipelines, Glue, Redshift, S3 etc., or equivalent cloud technologies. You have exceptional coding and design skills, particularly in Spark,…
