Key Duties and Responsibilities Architect and deliver large-scale data migration strategies for identity transformation Build robust batch and streaming pipelines using Apache Spark Develop data lineage, validation, and reconciliation frameworks to ensure data integrity Orchestrate highly parallel data workflows for performance and reliability Lead upgrades and optimisation of Spark and JVM environments in production Implement monitoring and observability to ensure system health and performance Design test frameworks to validate complex migration scenarios Lead & Elevate Engineering Teams Take ownership of critical components and drive engineering excellence Mentor team members through pair programming, reviews, and coaching Lead technical discussions and influence architecture decisions Proactively identify risks and remove delivery blockers Collaborate across teams to align technical solutions with business goals Technical Expertise: Your Tech Stack Languages: Scala, Java (Maven/Gradle) Big Data: Apache Spark, HDFS, S3, GCS Cloud: AWS, Azure, GCP, Databricks Containers: Kubernetes (EKS) Streaming: Kafka, Kinesis (bonus) Data Formats: Iceberg, Parquet, ORC, Avro (bonus) Observability: Prometheus, Grafana, Spark UI (bonus) Essential Skills, Experience, and Qualifications 8+ years’ experience in data engineering or software engineering Strong expertise in large-scale data migration and ETL pipelines Deep understanding of distributed systems and scalable architecture Hands‑on experience with cloud data platforms and containerised environments Knowledge of data security and privacy regulations (e.g., GDPR) A passion for building reliable, high‑performance systems Personal Attributes A natural leader and hands‑on builder Strong problem‑solver who thrives on complexity Collaborative mindset with a focus on knowledge sharing Comfortable challenging ideas and contributing to better outcomes Committed to quality, ownership, and continuous improvement Why This Role? Work on cutting‑edge, petabyte‑scale data projects Influence high‑value solutions across enterprise environments Grow through strong mentorship and leadership opportunities Be part of a culture built on trust, collaboration, and continuous learning Enjoy flexible working options and competitive rewards Hybrid- 2 days in-office. If you are excited by big data, complex systems, and meaningful impact, this is your opportunity to take your career to the next level. #J-18808-Ljbffr