Carbon Full Stack Engineer
As a Full Stack Data Engineer, you will be integral in developing and maintaining our next-generation digital analytics application. You will be building scalable web applications, designing robust data pipelines, and ensuring seamless data integration and processing. This role requires a unique blend of data engineering and software development expertise to design, develop, and maintain both the backend infrastructure and data pipelines that support our platform. You will work closely with cross-functional teams, including data analysts, Power BI developers, and UX/UI designers, to create a reliable and efficient data infrastructure that supports advanced analytics and reporting.
Principal Accountabilities
Develop and maintain robust front-end and back-end components of web applications, ensuring software quality, security, and maintainability through rigorous testing and code reviews.
Design, build, and maintain data pipelines to extract, transform, and load (ETL) data from various sources, optimizing data storage and retrieval processes for performance and scalability.
Design, develop, and maintain scalable data pipelines to extract, transform and load (ETL) data form various sources, optimizing data storage and retrieval processes for performance and scalability using Dagster.
Build data workflows that ensure high availability, reliability, and performance.
Integrate data from various sources (databases, APIs, event streams) into data lakes and warehouses.
Monitor, troubleshoot, and optimize data workflows for performance and scalability.
Create efficient data models and schemas to support the platform's analytical and operational needs using SQL and NoSQL databases.
Design, develop, and maintain RESTful APIs using industry standards to expose data and functionality to frontend applications.
Collaborate with data engineers, data analysts, and Power BI developers to integrate data sources into the analytics platform, providing technical guidance and mentorship to junior developers.
Deploy tested software applications and updates to production environments, troubleshoot and resolve issues related to data pipelines, application performance, and data integration, and implement monitoring and alerting mechanisms to ensure data pipeline reliability.