EU remote
Lead Data Engineer with Google cloud GCP (m/f/d)
About this role
Purpose As a lead data engineer, your purpose is to manage and oversee the entire data engineering process, from data acquisition to data analysis. You will be responsible for designing, building, and maintaining the data infrastructure necessary for effective data management and analysis. You will work closely with data scientists, analysts, and other stakeholders to ensure the accuracy, reliability, and accessibility of data.
Your role will also involve mentoring and leading a team of data engineers, setting technical direction, and ensuring the successful delivery of data engineering projects. About team/project You will join a newly formed team working on a greenfield data platform, building a modern data product from the ground up. The team is co-located, enabling strong collaboration, fast decision-making, and a supportive environment.
You will have real impact on architecture and ways of working from day one. The project leverages modern GCP technologies such as Google BigQuery, Google Cloud Dataflow, and Google Cloud Dataproc, offering interesting technical challenges without legacy constraints. WHAT WILL YOU DO? Design, build, and maintain scalable data pipelines using GCP services such as Google BigQuery, Google Cloud Dataflow, Google Cloud Storage, and Google Cloud SQL.
Develop and optimize ETL/ELT workflows for transforming and loading large datasets efficiently. Work with SQL and Python (including PySpark, Pandas, and NumPy) to process and analyze data. Build and maintain distributed data processing solutions using Spark and data parallelism principles. Collaborate with data scientists, engineers, and stakeholders to understand data requirements and deliver reliable solutions. Implement security best practices, including role-based access control, IAM policies, and data encryption.
Monitor, troubleshoot, and optimize system performance, identifying bottlenecks and improving efficiency. Contribute to data architecture and data modeling (e.g., star and snowflake schemas) where needed. Support best practices in version control and CI/CD pipelines (e.g., Git-based workflows). Mentor and guide junior data engineers, providing technical and professional development opportunities to help them grow in their careers.