Hays Specialist Recruitment is working in partnership with Security Scorecard to manage the recruitment of this positionThe end client is unable to sponsor or transfer visas for this position; all parties authorized to work in the US without sponsorship are encouraged to apply.This position is NOT eligible for subcontractors or those that require sponsorship.Hays is conducting an exclusive search for a Senior Data Engineer, for a Cybersecurity company based in NYC. Security Scorecard builds a very unique product that rates cybersecurity postures of corporate entities through the scored analysis of cyber threat intelligence signals for the purposes of third party management and IT risk management. They have a very modern Technology stack and work in a dynamic & agile environment.The position is a 100% remote and you'll be responsible for the management of the Analytic pipeline using Spark, Hardoop etc. Leverage cutting-edge technologies to support new and existing services and processes, drive projects through all stages of development and improving the effective output of the engineering team by managing quality and identifying inconsistencies. Your experience should involve 5+ years with Scala or another functional language (commercial environment preferred), 3+ years with Spark and the Hadoop ecosystem (or similar frameworks), Familiarity with tools like AWS and Docker, experience working with 3rd party software and Expert skills with SQLRemote Senior Data Engineer - Perm - New York, NY Remote Senior Data Engineer Skills & Requirements Responsibilities * Manage the analytic pipeline using Spark, Hadoop, etc * Leverage cutting-edge technologies to support new and existing and services and processes. * Quickly and efficiently design and implement in an agile environment * Work with other team members to implement consistent architecture * Drive projects through all stages of development * Actively share knowledge and responsibility with other team members and teams * Improve the effective output of the engineering team by managing quality, and identifying inconsistencies. Requirements: 3+ years of experience with: * Scala or Python, both preferred * Distributed systems (e.g. Spark, Hadoop) * Database systems (e.g. Postgres, MySQL) Experience with the following is preferred: * IP (v4/v6) allocation and addressing conventions * DNS conventions and best practices * Anti-abuse investigations * Bachelor's degree (CS, CE/EE, Math, or Statistics preferred) Why Hays? You will be working with a professional recruiter who has intimate knowledge of the Information Technology industry and market trends . Your Hays recruiter will lead you through a thorough screening process in order to understand your skills, experience, needs, and drivers. You will also get support on resume writing, interview tips, and career planning, so when there's a position you really want, you're fully prepared to get it. Additionally, if the position is a consulting role, Hays offers you the opportunity to enroll in full medical, dental or vision benefits. * Medical * Dental * Vision * 401K * Life Insurance ($20,000 benefit) Nervous about an upcoming interview? Unsure how to write a new resume? Visit the Hays Career Advice section to learn top tips to help you stand out from the crowd when job hunting. Hays is an Equal Opportunity Employer. Drug testing may be required; please contact a recruiter for more information.