The Role<ul><li>The Fastmarkets Data Platform team sits within the Technology function and is responsible for developing and delivering the company’s data strategy through data engineering, insights, analytics, and business intelligence. The team has built the organisation’s entire modern data platform from scratch and continues to evolve it to support Fastmarkets’ growth and AI ambitions.Reporting directly into the Head of Data Architecture, Engineering &amp; AI, we are looking for the right candidate to model and build our data pipelines and transformations. This is a fantastic opportunity to join us and contribute to a high-performing workstream delivering transformational change across both BI and our monetizable data assets. You will work with modern technologies and frameworks – with input to help deliver and adapt processes that enable a true “DataOps” environment and development experience.The Data Engineer will ensure that data is cleansed, tested, and delivered in accordance with Fastmarkets and industry best practice. In addition, the Data Engineer will enable monitoring via the observability platform and ensure that notifications and alerts are appropriately configured.We actively leverage AI tools across the data engineering workflow, including Anthropic Claude for documentation, code generation, and code review, as well as Snowflake Cortex AI capabilities for in-platform intelligence. The successful candidate will be expected to embrace AI-assisted development as a core part of how we work.The role requires a high standard of expertise in SQL and Python as well as specialist knowledge of modern approaches to delivering data projects using a code-first approach.</li></ul>Principal Accountabilities· Design, develop, and maintain scalable and efficient data pipelines from a wide variety of sources· Use dbt (Data Build Tool) to transform data through the various layers (Cleansed, Conformed, Presentation etc)· Ability to write custom connectors (Python) and leverage out-of-the-box data-loading tools· Operationalise enterprise data model by curating appropriate data models to service reporting, analytics, and data science use cases· Embed real-time, automated data quality checks, validations, exception handling, and alerting across all data pipelines· Work within Dagster as the primary orchestration and observability platform for all data pipelines· Manage CI/CD workflows using GitHub, GitHub Actions, and Kubernetes· Use zero-copy cloning and containerised on-demand development environments· Implement and review RBAC policies within Snowflake and related platforms· Embrace DataOps and a code-first approach to all data engineering work· Identify and promote best practices in data engineering; recommend improvements to existing processes and systems· Leverage AI-assisted development tools (Anthropic Claude, GitHub Copilot) to accelerate delivery and improve code quality· Excellent problem-solving skills and ability to embrace change· Effective communication and collaboration skills· Natural self-starter, with enthusiasm for learning and researchKEY INTERFACESInternal:· Head of Data Architecture, Engineering &amp; AI· Data Engineering team· Technology &amp; Infrastructure teams· Editorial and Pricing teams· BI &amp; Analytics teams &amp; stakeholders across the business· Product and Commercial teamsExternal:· Consultancy partners supporting platform delivery· Snowflake technical account team· Third-party vendor and tooling support

Our price data, forecasts, and market analyses give our customers a strategic advantage in complex, volatile and often opaque markets. Our events provide immersive experiences to network, trade and discuss the critical issues of the day.
Lithium Hangar Holdco Limited is the ultimate holding company for the Fastmarkets group of companies. Lithium Hangar Holdco Limited (14476245) is a company registered in England and Wales and is controlled by funds managed and/or advised by Astorg. Astorg is a European private equity firm with over &euro;20 billion of assets under management.

Senior Data Engineer

About the Job

About the Company

Fastmarkets Ltd

Similar Jobs