GPU Infrastructure Support Engineer

GPU Infrastructure Support Engineer

About Civo:

Civo is reimagining the cloud industry by providing developers and businesses with cutting-edge, developer-friendly cloud solutions. With a focus on simplicity, performance, and reliability, Civo enables its clients to unleash the full potential of their projects and scale with ease. As a company committed to excellence and innovation, Civo values diversity, collaboration, and continuous improvement.

About the Role:

GPU Infrastructure Support Engineer, you will serve as the frontline operational defense for Civo’s global GPU and AI infrastructure fleet. You will handle day-to-day customer tickets, platform alerts, hardware fault triage, and initial incident response across GPU nodes, high-speed networking fabrics, and Linux systems.

This is a hands-on, high-impact role requiring strong technical curiosity and structured troubleshooting discipline. You will own incidents end-to-end, conduct initial hardware diagnostics, coordinate smart-hands activities in our data centers, and ensure clean, evidence-rich handovers when escalating complex issues to Senior GPU Infrastructure Engineers.

Responsibilities:

24/7 Operations & Ticket Triage

• Incident Triage: Monitor platform dashboards and queue systems to acknowledge, prioritize, and triage customer-reported issues within strict SLA targets.

• Customer Communication: Provide clear, professional, and technical status updates to enterprise clients via ticketing channels and status pages.

• Shift Handovers: Maintain accurate event logs and execute clean, detailed shift handovers to ensure continuous operational coverage.

GPU Node & Hardware Triage

• First-Line Diagnostics: Execute runbooks to inspect node health using nvidia-smi, NVIDIA Data Center GPU Manager (DCGM), and system event logs (dmesg, journalctl).

• Out-of-Band Management: Access and manage bare-metal host BMCs via IPMI or Redfish to perform power cycles, remote console inspections, and system event log extractions.

• RMA Preparation: Gather diagnostic logs and serial numbers to build complete, evidence-rich vendor RMA submission packages for failed GPUs, DIMMs, or storage drives.

Data Center & Field Support Coordination

• Smart-Hands Coordination: Act as the primary technical contact for local data center facility technicians, directing physical component reseats, optic replacements, and cable testing.

• Asset Tracking: Maintain absolute data integrity across Civo’s DCIM and IPAM systems (e.g., NetBox) during hardware swaps or maintenance.

Platform & Environment Troubleshooting

• User Support: Assist customer teams with basic SSH access, GPU device passthrough checks, Slurm job queue status, and container runtime issues.

• Runbook Execution & Improvement: Follow operational runbooks for routine maintenance and propose incremental updates based on real-world incident trends.

Key Results/Objectives:

• SLA Adherence: Maintain >98% compliance on initial response and update SLAs across all priority tiers.

• First-Contact Resolution: Resolve >= 60% of tier-1 platform and access tickets without requiring L3 escalation.

• RMA Processing Speed: Package and submit verified RMA evidence to hardware vendors within 4 hours of fault confirmation.

Requirements:

Experience & Core Qualifications

• 2–4+ years of technical experience in an IT infrastructure support, service desk, or data center operations role within a structured, customer-facing environment.

• Demonstrated hands-on experience troubleshooting server hardware and Linux operating systems.

Technical Expertise

• Linux Administration: Strong CLI proficiency with systemd, file permissions, storage mounts, and network diagnostic utilities (ip, ping, traceroute, netstat).

• GPU & Hardware Diagnostics: Working knowledge of NVIDIA GPU server architectures, including command-line tools like nvidia-smi and basic DCGM health checks.

• Hardware & Out-of-Band: Familiarity with server architecture (PCIe, NVMe, memory topology) and Out-of-Band management (iDRAC, iLO, BMC via IPMI/Redfish).

• Ticketing & ITIL: Working understanding of incident management workflows, prioritization, and ticket lifecycle tracking.

Soft Skills

• Exceptional written communication skills with the ability to summarize complex technical logs into clear customer updates and internal escalation notes.

• Strong organizational discipline and a methodical approach to step-by-step technical isolation.

Nice to Have:

• Experience with Slurm workload manager or Kubernetes (kubectl basics).

• Basic awareness of high-performance networking concepts (InfiniBand, RoCEv2, RDMA).

• Linux certifications (RHCSA, CompTIA Linux+) or Kubernetes foundations (CKA/KCNA).

Why Join Civo?

• Competitive compensation and benefits package.

• 4-day work week company (unless attending an event).

• Uncapped holiday.

• Remote work environment with flexibility and autonomy.

• Collaborative and inclusive culture that values diversity and creativity.

• Opportunity to work with a dynamic and innovative team in the fast-growing cloud industry.