We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage with us to do the straightforward things, they ask us to help on their biggest challenges which means we get to work with a wide range of tools and technologies and there are always new things to learn.

Data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software engineering with the knowledge and experience of building data solutions in order to deliver business value.

Job Responsibilities:

• As a data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps culture.
• You’ll work in a fast moving, agile environment, within multi-disciplinary teams of highly skilled consultants, delivering modern data platforms into large organisations.
• You can expect to get involved in variety of projects in the cloud (AWS, Azure, GCP), learning about and using data services such as Databricks, Data Factory, Synapse, Kafka, Redshift, Glue, Athena, BigQuery, S3, Cloud Data Fusion etc.

Requirements:

• You’re an engineer at heart and enjoy the challenge of building reliable, efficient data applications systems, services and platforms.
• You have a good understanding of coding best practices and design patterns and experience with code and data versioning, dependency management, code quality and optimisation, error handling, logging, monitoring, validation and alerting.
• You have experience in writing well tested object-oriented python.
• You have experience with using CI/CD tooling to analyse, build, test and deploy your code.
• You have a good understanding of design choices for data storage and data processing, with a particular focus on cloud data services.
• You have experience in using parallel computing to process large datasets and to optimise computationally intensive tasks.
• You have experience in programmatically deploying, scheduling and monitoring components in a workflow.
• You have experience in writing complex queries against relational and non-relational data stores.