Skip to content

Data Engineer

  • On-site
    • Johannesburg, Gauteng, South Africa

Job description

About the Company: 

Khonology is a digital services company focused on software development, Application Support, data analytics and engineering.

We are looking for a skilled Data Engineer to join our team. The ideal candidate will have a strong background in Python, AWS/Azure Cloud technologies, data pipeline management, ETL and ELT principles, and Big Data technologies. You will be responsible for designing, building,
and maintaining scalable and robust data systems to support our data-driven initiatives and productionizing data related systems.

Core Responsibilities and Competencies:

  • Programming: Write efficient and maintainable code in Python and SQL.

  • Data Pipeline Management: Design, develop, and maintain data pipelines to ensure smooth data flow from various sources to the data warehouse.

  • Production Systems: Apply best practices in production systems management to ensure reliability and scalability.

  • ETL and ELT Processes: Implement and optimize ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes to ensure data is correctly ingested, transformed, and loaded.

  •  Data API Integration: Develop and maintain data APIs to facilitate seamless data access and integration with other systems.

  • Big Data Technologies: Utilize Hadoop and Spark to process large volumes of data efficiently.

  • Database Management: Manage and optimize relational databases to ensure high performance and reliability.

  • Data Modeling: Design and implement data models to support data product and
    machine learning needs.

  • Query Optimization: Ensure that queries are always optimized for performance to enhance system efficiency.

Job requirements

Our current tech stack:

The ideal candidate should be familiar with our current tech stack:

  • Could Platforms : AWS, Microsoft Azure

  • Main Programming language: Python, SQL (Domain-Specific Language)

  • Big Data technologies (On prem): Hadoop, Spark

  • Code Repositories: BitBucket, GitHub, CodeCommit

  • Deployment frameworks: Bamboo, Octopus Deploy, Codebuild and CodeDeploy

or