Remote
[Expression of Interest] Data Engineer
About this role
Role Summary We are looking for a talented and driven engineer to join the Ascend Sandbox team — Experian's cloud-based analytical platform that empowers clients and data scientists to explore, model, and derive insights from data in a secure, governed environment. You will play a key role in building, maintaining, and scaling the platform that serves both internal teams and external clients across the APAC region. What you’ll be doing Design, build, test, deploy and operate scalable batch and, where applicable, streaming data pipelines for the Ascend Sandbox analytical platform.
Develop high-quality Python, PySpark and SQL solutions to ingest, transform and process large-scale datasets with accuracy, reliability and efficiency. Design data models and reusable, trusted datasets/data products for analytics, data science and downstream applications. Engineer and optimise AWS-based data solutions using relevant services such as Amazon S3, AWS Glue, Amazon Athena, Amazon Redshift and Amazon EMR, according to the platform architecture.
Apply data lake and lakehouse engineering practices, including efficient storage formats, partitioning, schema evolution and storage/query optimisation. Design robust ETL/ELT patterns, including incremental processing, dependency management, idempotency, retries, backfills and failure recovery. Implement workflow orchestration and scheduling for data pipelines using the platform's approved tooling, such as Airflow/MWAA, AWS Step Functions or AWS Glue Workflows.
Build automated data quality controls and monitoring for completeness, validity, reconciliation, schema compliance, freshness, pipeline health and processing SLAs/SLOs. Optimise Spark workloads and distributed data processing for performance, scalability and cost, including partitioning, joins, shuffles, skew and resource utilisation. Implement practical data governance, metadata, lineage, cataloguing, security and access-control requirements across data pipelines and datasets.
Collaborate with clients, business users, data scientists and technical teams to translate business and data requirements into scalable data-processing solutions and integrations. Investigate and resolve production data and pipeline issues, perform root-cause analysis, and implement preventive improvements to reliability and operational efficiency. Ensure data engineering solutions comply with established architecture, coding, testing, security and quality standards, and contribute to code reviews, CI/CD and automated deployment practices.