About the job Intermediate Data Engineer Responsibilities: Develop algorithms to transform data into useful, actionable information Design, construct, install, test and maintain highly scalable data management systems Ensure systems meet business requirements and industry practices Build high-performance prototypes and proof of concepts Research opportunities for data acquisition and new uses of existing data Develop data set processes for data modelling, mining and production Integrate new data management technologies and software engineering tools into existing structures Create custom software components (e.g. specialized UDFs) and analytics applications Employ a variety of languages and tools (e.g. scripting languages) to marry systems together Install and update disaster recovery procedures Recommend ways to improve data reliability, efficiency and quality Support the achievement of the business strategy, objectives and values Participate and support corporate responsibility initiatives for the achievement of business strategy Build, test, and maintain database pipeline architectures Building data products for the data analytics team to ingest and train models on Ensure data systems (e.g. Clickhouse, MySQL) are functioning correctly and performing to expectations Collaborate with management to understand company objectives Create new data validation methods and data analysis tools Ensure compliance with data governance and security policies Document all created artefacts, create ERDs and any other needed documentation to make sure data analytics are working at capacity Explore new technology and look into new ways of reaching specific objectives. Work closely with our in-house team as well as outsourced service providers and partners. Requirements: Degree / Diploma in Data Engineering preferred but not required. 2+ years of cloud experience 3+ years of experience building, maintaining and administering a DBMS, such as MySQL or Postgres. 2+ years experience in SQL development, including writing CTEs, and creating schemas and tables. A successful history of manipulating, processing and extracting value from large disconnected datasets. Strong analytic skills related to working with unstructured datasets. Python ORM programming experience (SQLalchemy) would be a bonus Proficiency in both written and spoken English A fundamental understanding of Basic Machine Learning Knowledge about server and data security on the web #J-18808-Ljbffr