Solutions/Databricks Architect
Please mention DailyRemote when applying
AI Summary
The Databricks Architect will design, implement, and optimize scalable data lakehouse platforms tailored for federal data solutions. They will lead the integration of Databricks with federal cloud services while ensuring compliance with security standards like FedRAMP and FISMA.
Benefits:
- 401(k) matching
- Dental insurance
- Health insurance
- Paid time off
- Parental leave
- Vision insurance
Role Description:
The Databricks Architect will design, implement, and optimize our enterprise data lakehouse platform. The ideal candidate will be a technical leader with deep expertise in Databricks, cloud data architectures, and modern data engineering practices.
Key Responsibilities:
- Design and implement scalable data lakehouse architectures using Databricks for federal data solutions.
- Develop and optimize end-to-end data pipelines, ensuring alignment with federal performance, security, and compliance requirements. Implement data governance and security protocols in accordance with federal standards (e.g., FedRAMP, FISMA, NIST, etc.).
- Collaborate closely with federal stakeholders to gather and analyze requirements and translate them into actionable data solutions. Lead efforts to integrate Databricks with federal cloud services, including AWS GovCloud, Azure Government, or other federal platforms.
- Provide subject matter expertise on Databricks capabilities, including Delta Lake, Databricks SQL, and Apache Spark.
- Optimize Databricks clusters for cost efficiency, high availability, and performance in a secure environment.
- Enable advanced analytics and machine learning workflows for federal use cases, leveraging tools such as MLflow and related frameworks.
- Develop and implement real-time data processing solutions for mission-critical federal applications.
- Stay updated on federal data policies and ensure solutions align with relevant standards and compliance requirements.
Minimum Skills:
- 5–10 years of experience in data engineering, data architecture, or distributed data processing, with at least 3+ years of hands-on experience with Databricks.
- Proficiency in Databricks, including Databricks SQL, Delta Lake, and the Lakehouse platform.
- Expertise in Apache Spark and distributed data processing frameworks.
- Advanced knowledge of Python, Scala, or Java for data engineering and analytics.
- Hands-on experience with cloud platforms designed for federal use, such as AWS GovCloud, Azure Government, or Google Cloud Public Sector.
- Strong knowledge of ETL processes, data workflows, and data transformation pipelines.
- Strong understanding of data lakes, data warehouses, data modeling, and schema design.
- Experience implementing data governance and security protocols within regulated environments.
- Knowledge of federal compliance standards, such as FISMA, FedRAMP, NIST 800-53, and CMMC (Cybersecurity Maturity Model Certification).
- Experience in securing data architecture for federal agencies, including encryption and access control best practices.
- Advanced expertise with SQL for querying and managing large datasets in relational databases.
- Familiarity with NoSQL databases and their use cases.
- Proven ability to lead cross-functional teams in the delivery of complex data solutions.
- Strong interpersonal and communication skills to engage with federal stakeholders and align technical solutions to mission objectives.
- Experience training and mentoring technical teams on Databricks best practices.
Preferred Skills
- Expertise in CI/CD pipelines and deployment of Databricks workflows and notebooks.
- Experience with Infrastructure-as-Code (IaC) tools such as Terraform, CloudFormation, or ARM templates.
- Strong understanding of machine learning pipelines and operationalizing models in Databricks MLflow.
- Familiarity with AI/ML frameworks like TensorFlow, PyTorch, or Scikit-learn.
- Experience with real-time data streaming using Kafka, Azure Event Hubs, or AWS Kinesis.
- Knowledge of other Big Data tools and frameworks such as Hadoop, Flink, or Elasticsearch.
- Experience with real-time analytics and event-driven architectures in mission-critical systems.
- Ability to work in highly regulated environments with federal stakeholders.
- Expertise in managing multiple priorities, meeting deadlines, and adapting to evolving client needs.
- Experience in preparing technical documentation, reports, and presentations tailored to federal audiences.
- Advanced Databricks certifications (e.g., Databricks Certified Professional Data Engineer).
- Certifications in federal cloud and cybersecurity standards (e.g., AWS Certified Security – Specialty, CCSP, CISSP, or CISM).
- Public Trust (or higher) clearance eligibility; prior federal client experience.
This is a remote position.
Automatically Apply to the Best Remote Jobs
Stop the endless job search. Our AI finds and applies to the best jobs for you.
