Data Engineer- SC- Contract until June 2022- Remote

Apply for this position Please mention DailyRemote when applying
Posted 3 days ago Worldwide Salary undisclosed
Before you apply - make sure the job is legit.

Attempting to apply for jobs might take you off this site to a different website not owned by us. Any consequence as a result for attempting to apply for jobs is strictly at your own risk and we assume no liability.

Job Description

Data Engineer- SC- Contract u ntil June 2022- Remote

We are currently recruiting for an experienced Data Engineer to join a government department

You need an active SC for this role

Key Responsibilities

You will development of ETL methods for a range of internal and external data sources converting these from unstructured formats to dynamic tables and views in Hive and Impala;

Supporting the continuing development and transformation of data, making these available to core users in near Real Time;Providing coding support and coaching to a growing team of data engineers sharing best practices and established methods.

Assisting in the development of analytical layers of data from raw HDFS files for use in producing a range of outputs.

Key Skills

Extensive proven experience of data engineering and architectural techniques, including data wrangling, data profiling, data preparation, metadata development, and data upload/download;

Proven experience of big data environments, including the Hadoop Stack (Cloudera), including data ingestion, processing, and storage using HDFS, Spark, Hive, and Impala;

Extensive hands-on experience in developing ETL functionality in a cloud or on-premise environment;

Experience of using tools such as python and SQL (in Spark) to profile, query and structure large-volume data;

Proven experience of using Cloud Services particularly in the context of Hadoop;

Experience in developing/utilising programming and query languages eg SQL (Hive Impala specifically), Python (through Spark), Scala.

Capacity to develop code and ETL processes outside of systems, using own equipment where necessary.

- SmartSourcing provides services as an Employment Agency and welcomes applications from all suitably qualified people regardless of age, race, religion, disability, age, gender or sexual orientation.