Staff Software Engineer, AI Inference Gateway
You will own the end-to-end AI Gateway system, including request routing, streaming, and horizontal scaling across the infrastructure. Additionally, you will build and tune fleet-efficiency algorithms to optimize throughput and performance for inference nodes.