Job Title: AWS Big Data Engineer
Duration: 3 months Contract to Fulltime
Location: Full Remote
** This is basically a cloud development role, thus looking for an Engineer with some architecture background who is heavily focused on hands-on Development with Python, Spark and Glue**
- 3+ years of software development in Python
- Experience with Python, Spark, Glue and EMR
- 2+ years of AWS Data Engineering experience
- Collabera is looking to hire an experienced and highly motivated AWS Big Data engineer to design and develop data pipelines using AWS Big Data tools and services and other modern data technologies.
- In this role, they will play a crucial part in shaping the future big data and analytics initiatives for many customers for years to come.
- Who is passionate about building at scale on (AWS).
- Thrive at simplifying hard problems and can articulate the solution to both technical and non-technical stakeholders.
- Build end-to-end big data pipelines on AWS, including:
- Ingestion/replication from traditional on-prem RDBMS (e.g. Oracle, MS SQL Server, MySQL, Postgres) to AWS
- Streaming ingestion with Kinesis Streams, Kinesis Firehose, and Kinesis Analytics
- Change Data Capture (CDC) logic and partitioning
- ETL and Analytics with AWS Glue, Glue Streaming, EMR, Spark, Presto, Athena, Flink, Python, PySpark
- Refactoring of existing RDBMS scripts (e.g. PL/SQL. T-SQL, PL/pgSQL) to PySpark jobs
- Buildout of data warehouse and published data sets using RedShift, Aurora, RDS, ElasticSearch
- Python scripting with AWS Lambda
- 3+ years of experience in software development with Python
- 2+ years of development experience with Spark/PySpark, Pandas
- 3+ years of database development experience with RDBMS, including development of stored procedures
- 2+ years of ETL development experience
- 2+ years of hands-on data engineering on AWS, including S3, Kinesis, Glue, Athena, RDS/Aurora, RedShift
- AWS Solutions Architect Professional and Data Analytics Specialty (formerly Big Data Specialty) Certifications are a plus
- A Bachelor's Degree from an accredited college in Computer Science or equivalent experience
python,spark,glue,EMR,AWS,Engineer,Development,Architecture,Data ,data pipelines - provided by Dice