Senior PySpark Developer
PySpark Developer with experience working with EMR/AWS, to migrate SAS models into AWS. This position will directly support and work as a hands-on technical contributor to our clients team. Ideal candidates would have working knowledge of AWS. EMR, and SAS, ETL, PySpark. Experience with Agile methodology a plus.
The main objectives for this role:
- Application design that meets requirements (i.e. performance and business)
- Hands on conversion to PySpark (able to run SAS and reconcile with SAS Data. Be able to quickly learn how to read SAS based on running SAS and reading SAS documentation)
- Guide other experienced developers
Additional Responsibilities include:
- Work with business and technical leads on the Transformation of critical Credit Risk Forecasting applications to Microservices AWS EMR Architecture leveraging PySpark.
- Provide guidance on Application Architecture, design and implementation for forecast process for credit loss and home price for Fortune 100 client.
- Work with, lead and mentor the business and technology teams. Build a solution based on cloud, microservices API's. Decompose legacy monolithic systems into Microservices and API's Running in the cloud.
- Highly visible position interacting directly with executive-level clients
- Work in lock step with project management to navigate complex customer landscapes
- Ability to look across a variety of data sources to devise proven yet inventive ways of organizing, consolidating, and optimizing how the data is used and what it means
- Collaborate as part of any project team - large or small
- Take on any task, and capable of clearly communicating with their customer's, management and other key stakeholders in both technical and high-level terms
- Proven ability to document, promote and present your work to other teams and to customers
- Collaborate across internal practices and with other team members
- Work within an organized and structured delivery-centric team
- Translate customer requirements and needs into proven design patterns
- 5+ years of experience working in a PySpark/EMR environment
- As Lead Developer, expected to be hands on with converting SAS Code to PySpark ensuring quality and maintainability.
- Well versed in EMR PySpark Performance tuning leverage EMR Architecture (Master, Cluster, memory utilization etc.)
- Understanding of AWS computing and data architecture to create a sound and high performing implementation for Credit Expense Forecasting Process.
- Experience with Parallel programming with Big Data, ETL, and forecasting models
- Designing API layer using tools like Apigee, AWS API Gateway, API Security, API Framework, and the like.
- Proficiency with one or more programming languages (Java, Python, PySpark preferred)
- Proficiency in SQL, relational and non-relational databases, query optimization, and data modeling.
- Experience working in a Linux-based environment.
- Experience working on teams operating under an Agile Scrum delivery methodology
- Experience in Financial Services/Secondary Mortgage industry is a plus
- Demonstrated ability to manage multiple priorities and deadlines
- Ability to work independently or as part of a team
- provided by Dice