Sr. Data Engineer

Apply for this position Please mention DailyRemote when applying
timePosted 4 days ago location United States salarySalary undisclosed
Before you apply - make sure the job is legit.

Attempting to apply for jobs might take you off this site to a different website not owned by us. Any consequence as a result for attempting to apply for jobs is strictly at your own risk and we assume no liability.

Job Description

Sr. Data Engineer with AWS. Location Newark, CA. Duration Long Term. We are looking for a Senior Data Engineer, who is looking for a challenge, enjoys thinking big and looking to make their mark on an extremely fast growing company. If you have hands-on experience designing and developing streaming and IoT data pipelines this role is for you. Be part of a group who will be building large and building fast, working with a very talented team of engineers and collaborating with the brightest mind in the Automotive industry. The Role Hands-on design and develop streaming and IoT data pipelines. Developing streaming pipeline using MQTT, Kafka, Spark Structure Streaming Orchestrate and monitor pipelines using Prometheus and Kubernetes Deploy and maintain streaming jobs in CICD and relevant tools. Python scripting for automation and application development Design and implement Apache Airflow and other dependency enforcement and scheduling tools. Hands-on data modeling and data warehousing Deploy solution using AWS, S3, Redshift and DockerKubernetes Develop storage and retrieval system using Presto and ParquetORC Scripting with Apache Spark and data frame. Qualifications Bachelor or Masters in Software Engineering or Computer Science 6 - 8+ years of experience in Data Engineering and Business Intelligence. Proficient in IoT tools such as MQTT, Kafka, Spark Proficient with AWS, S3, Redshift Experience with Presto and ParquetORC Proficient with Apache Spark and data frame. Experienced in containerization, including Docker and Kubernetes Expert in tools such as Apache Spark, Apache Airflow, Presto Expert in design and implement reliable, scalable, and performant distributed systems and data pipelines Extensive programming and software engineering experience, especially in Java, Python, Experience with Columnar database such as Redshift, Vertica Great verbal and written communication skills.