Job Description Summary of Role The purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data products, and support data initiatives. In addition, Data Engineers enhance the data infrastructure of the to enable advanced analytics, machine learning and artificial intelligence by providing clean, usable data to stakeholders. They also create data pipelines, Ingestion, provisioning, streaming, self-service, API and solutions around big data that support the strategy to become a data driven organisation. Responsibilities Maintain data pipelines on a day-to-day basis, supporting data squads with tasks such as data profiling, data cleaning, data configuration, data support, data validation and data quality assurance. Support and maintain the data infrastructure to ensure it is secure, available and reliable. Build and maintain data pipelines covering ingestion, provisioning, streaming and API, progressing from data virtualisation to data ingestion and ultimately to data provisioning. Monitor and run data pipelines, configure and support minor changes, and ensure successful operation. Create virtual data bases and produce data extracts for business needs. Collaborate with Data Analysts to perform data profiling, validation and documentation for Epics. Monitor data pipelines and infrastructure, provide first‑line support, resolve issues and meet SLA timelines for data availability and warehouse reliability. Monitor and manage cloud compute and storage processes daily to keep cloud pipelines running successfully. Run daily operational reports, ensure all jobs ran successfully and keep the data warehouse aligned with standards. Work collaboratively with business stakeholders to gain business knowledge, understand data extracts and optimise business queries. Qualifications Matric and a Bachelor's degree in Computer Science, Information Systems, Engineering, or related field. Seven years of experience in data engineering or a related discipline. Strong experience with SQL and relational database technologies. Proficiency with ETL/ELT tools and frameworks such as Airflow, DBT, Azure Data Factory, and AWS Glue. Experience with cloud platforms Azure, AWS or GCP. Knowledge of big data technologies such as Spark and Hadoop. Experience with Python or Scala for data processing. Skills in SQL, Python (required), Cloud services (Azure/AWS/GCP), Data warehousing (Snowflake, Redshift, BigQuery, Synapse), Big data frameworks (Spark, Databricks), Version control (Git), Containerisation (Docker, Kubernetes). Salt is acting as an Employment Business in relation to this vacancy. #J-18808-Ljbffr
Data Engineer
SALT DIGITAL RECRUITMENT
johannesburg, johannesburg
Published 4 days ago
Report job