Job Search / Engineers

Big Data (Hadoop) Engineer

Job Details

Job Type
Jackson Bruce
11 days ago
Our client has a requirement for the services of an experienced Big Data (Hadoop) Engineer to facilitate the development and deployment of new capabilities within a large new multi-year program within their Technology & Innovation (T&I) section.

The Services, delivered in collaboration with and under supervision by nominated personnel, include, but are not limited to, the following:
  • Provide best practice advice and technical leadership on implementation of data streams within a Hadoop platform.
  • Development of data streams using Apache NiFi, Spark and Kafka (or equivalent technologies).
  • Vendor engagement/product evaluation and on-going maintenance of infrastructure deployed at regional sites
  • Provide technical direction / advice to vendors performing professional services on behalf of the client
  • Assist T&I staff with implementing new solutions into the ICT environment.
  • Work effectively in a small team environment
  • Provide knowledge/skills transfer to other team members
  • Record and maintain files/information in accordance with client's requirements

Mandatory Criteria
  • Proven experience with the development of data streaming and processing pipelines using Apache NiFi, Apache Kafka, Apache HBase, HDFS, and Apache Spark.
  • Data development and management experience including but not limited to data migration, modelling and analytics.
  • Proven experience implementing role based access control within a analytics platform.
  • Proven DevOps experience in large infrastructure environment, big data platform experience including but not limited to administration and support for a Hortonworks/Cloudera/Hadoop platform.
  • Proven system implementation and integration skills, including but not limited to; systems configuration and change management process and documentation.
  • Demonstrated ability to communicate and work effectively within a team

Highly Desirable
  • Expertise in the following technologies:
  • RHEL Linux (or similar) administration and support experience
  • Graph databases (Neo4J).
  • JSON, Java, Elasticsearch
  • Database administration and management experience including but not limited to Microsoft SQL, Oracle and NoSQL technologies.
  • Experience working in the government sector especially with agencies that run sensitive solutions
  • Experience with containerisation technologies
  • Automation experience in private or public cloud, relevant qualifications and certifications.
  • Experience working in Agile Development environments.
  • Experience with package management tools and practices (e.g. SNYK).

If you're interested in finding out more, please contact Jackson Bruce on 3031 4519 OR email OR hit "Apply Now"!