A well-funded AI infrastructure firm building realtime systems at massive scale.
Senior Machine Learning Engineer, Inference
Referral bonus eligible. Know someone who would be great for this role? Refer them to a Fluency Digital recruiter. If they're placed, you may be eligible for a referral bonus of up to $5,000 (terms apply). Refer someone for this role
Current generative models often stumble when forced to respond instantly under heavy load, creating latency that breaks immersion. You will architect the high-performance serving layer that allows complex neural networks to operate with millisecond precision across global clusters. This position demands deep expertise in optimizing compute kernels and managing containerized workloads to ensure seamless user experiences.
What they're looking for
- Five or more years of experience shipping production ML systems with a focus on inference optimization.
- Expert-level proficiency in C++, Rust, or Go alongside strong Python scripting abilities.
- Demonstrated success scaling PyTorch models within Kubernetes and Docker environments.
- Deep understanding of memory management, concurrency, and low-latency system design.