EU remote
Big Data Developer
About this role
About the role We are looking for a Big Data Developer to join an Agile environment and work on the design, development and evolution of data lake solutions for Market Risk use cases . You will work with technologies such as Spark, Scala and SQL , developing large-scale data processing pipelines and contributing to the ingestion, transformation, validation and publication of critical data. The role combines hands-on technical development with problem-solving and collaboration across Agile squads, functional teams and production support teams.
What will you do? Design, develop and maintain data lake pipelines using Apache Spark, Scala and SQL . Build and maintain scalable data processing solutions for large volumes of data. Orchestrate batch workflows and dependencies using tools such as Control-M and Airflow . Implement data ingestion, transformation, validation and publication processes for Market Risk use cases. Analyse and optimise the performance, reliability and scalability of Spark-based workloads.
Analyse technical and functional requirements, identify gaps and propose effective solutions. Contribute to delivery planning and collaborate with different teams to ensure successful implementation. Support production incidents , perform root-cause analysis and contribute to continuous improvement of operational processes. Work closely with Agile squads, functional teams and production support teams to deliver high-quality solutions and business value.
Follow software engineering and data development best practices to ensure robust, maintainable and production-ready solutions. What are we looking for? Mandatory requirements: 2+ years of professional experience in Big Data, Data Lakes or cloud-based data platforms. Strong hands-on experience with Apache Spark, Scala and SQL for large-scale data processing. Good understanding of Data Lake architecture , data quality controls, data lineage and production-grade data pipelines.
Strong analytical and problem-solving skills. Excellent communication and teamwork skills, with the ability to collaborate effectively with technical and functional stakeholders. Adaptability and a proactive approach to solving technical challenges. English: C1 level or equivalent professional fluency. Spanish is considered a plus. Nice to have Experience with any of the following will be highly valued: Workflow orchestration and scheduling tools, particularly Control-M and Airflow .