Solutions/Databricks Architect
Please mention DailyRemote when applying
AI Summary
The Solutions/Databricks Architect will design and implement scalable data lakehouse architectures and optimize end-to-end data pipelines for federal data solutions. They will also collaborate with federal stakeholders to ensure compliance with security standards and enable advanced analytics and machine learning workflows.
PCI Professional Services is seeking a Solutions/Databricks Architect for an upcoming opportunity. This role is remote and reports to the Program Manager. The ideal candidate will be a technical leader with deep expertise in Databricks, cloud data architectures, and modern data engineering practices.
Responsibilities:
- Design and implement scalable data lakehouse architectures using Databricks for federal data solutions.
- Develop and optimize end-to-end data pipelines, ensuring alignment with federal performance, security, and compliance requirements. Implement data governance and security protocols in accordance with federal standards (e.g., FedRAMP, FISMA, NIST, etc.).
- Collaborate closely with federal stakeholders to gather and analyze requirements and translate them into actionable data solutions. Lead efforts to integrate Databricks with federal cloud services, including AWS GovCloud, Azure Government, or other federal platforms.
- Provide subject matter expertise on Databricks capabilities, including Delta Lake, Databricks SQL, and Apache Spark.
- Optimize Databricks clusters for cost efficiency, high availability, and performance in a secure environment.
- Enable advanced analytics and machine learning workflows for federal use cases, leveraging tools such as MLflow and related frameworks.
- Develop and implement real-time data processing solutions for mission-critical federal applications.
- Stay updated on federal data policies and ensure solutions align with relevant standards and compliance requirements.
Minimum Skills
- Bachelor's degree in a related discipline
- 5–10 years of experience in data engineering, data architecture, or distributed data processing, with at least 3+ years of hands-on experience with Databricks.
- Public Trust (or higher) clearance eligibility
- Proficiency in Databricks, including Databricks SQL, Delta Lake, and the Lakehouse platform.
- Expertise in Apache Spark and distributed data processing frameworks.
- Advanced knowledge of Python, Scala, or Java for data engineering and analytics.
- Hands-on experience with cloud platforms designed for federal use, such as AWS GovCloud, Azure Government, or Google Cloud Public Sector.
- Strong knowledge of ETL processes, data workflows, and data transformation pipelines.
- Strong understanding of data lakes, data warehouses, data modeling, and schema design.
- Experience implementing data governance and security protocols within regulated environments.
- Knowledge of federal compliance standards, such as FISMA, FedRAMP, NIST 800-53, and CMMC (Cybersecurity Maturity Model Certification).
- Experience in securing data architecture for federal agencies, including encryption and access control best practices.
- Advanced expertise with SQL for querying and managing large datasets in relational databases.
- Familiarity with NoSQL databases and their use cases.
- Proven ability to lead cross-functional teams in the delivery of complex data solutions.
- Strong interpersonal and communication skills to engage with federal stakeholders and align technical solutions to mission objectives.
- Experience training and mentoring technical teams on Databricks best practices.
Preferred Skills
- Expertise in CI/CD pipelines and deployment of Databricks workflows and notebooks.
- Experience with Infrastructure-as-Code (IaC) tools such as Terraform, CloudFormation, or ARM templates.
- Strong understanding of machine learning pipelines and operationalizing models in Databricks MLflow.
- Familiarity with AI/ML frameworks like TensorFlow, PyTorch, or Scikit-learn.
- Experience with real-time data streaming using Kafka, Azure Event Hubs, or AWS Kinesis.
- Knowledge of other Big Data tools and frameworks such as Hadoop, Flink, or Elasticsearch.
- Experience with real-time analytics and event-driven architectures in mission-critical systems.
- Ability to work in highly regulated environments with federal stakeholders.
- Expertise in managing multiple priorities, meeting deadlines, and adapting to evolving client needs.
- Experience in preparing technical documentation, reports, and presentations tailored to federal audiences.
- Advanced Databricks certifications (e.g., Databricks Certified Professional Data Engineer).
- Certifications in federal cloud and cybersecurity standards (e.g., AWS Certified Security – Specialty, CCSP, CISSP, or CISM).
- Prior federal client experience.
PCI Federal (PCIF) and its subsidiaries are an equal-opportunity employer. PCIF does not discriminate on the basis of age, sex, race, national origin, religion, marital status, sexual orientation or identity, Veterans or Disability status.
Preference may be extended to qualified Native American Indian candidates in accordance with applicable federal law.
Automatically Apply to the Best Remote Jobs
Stop the endless job search. Our AI finds and applies to the best jobs for you.
