San Francisco startup building AI systems for instant, interactive generated media
Infrastructure Engineer — Managed Inference
Referral bonus eligible. Know someone who would be great for this role? Refer them to a Fluency Digital recruiter. If they're placed, you may be eligible for a referral bonus of up to $5,000 (terms apply). Refer someone for this role
You will own the inference platform that serves real-time generative models to millions of users. Your mornings start with Prometheus dashboards — spotting latency regressions in a Kubernetes fleet spanning AWS and GCP regions, then shipping Terraform changes to isolate a noisy-neighbor GPU workload before lunch. The models change weekly; your systems must absorb new architectures without hand-holding. This is infrastructure work where milliseconds matter and 'works on my cluster' is not in the vocabulary.
What they're looking for
- 5+ years running production ML infrastructure at scale, with deep experience in Kubernetes scheduling for GPU workloads
- Proven track record reducing inference latency and cost on cloud GPU instances (AWS or GCP)
- Expert-level fluency in Terraform, Docker, and observability stacks (Prometheus/Grafana or equivalents)
- Strong grasp of PyTorch model serving patterns: torch.compile, tensor parallelism, continuous batching
- Comfort with on-site collaboration in San Francisco; early-employee mindset without the title