AI Infrastructure Engineer role owning Netpreme's inference-serving stack end to end, including tuning vLLM, SGLang, and TensorRT-LLM and architecting distributed serving with NVIDIA Dynamo, Kubernetes, Helm, and Ansible. The work covers benchmarking latency and throughput, parallelism and KV-cache optimization, and collaborating with hardware and systems engineers. Netpreme is a Cambridge, MA and Santa Clara, CA company building memory processing silicon (X-Mem) founded by MIT researchers.
The biggest bottleneck in AI systems today is memory. Chips compute faster than they can be fed data—a problem known as the memory wall—and Netpreme is building the solution that allows AI memory to scale independently of AI compute. Our approach spans the full stack, from a proprietary memory management framework at the XPU kernel and OS levels to X-Mem™, the world’s first Memory Processing Unit (MPU) silicon. Founded by MIT researchers at the intersection of optical networking, ML systems, and GPU kernel optimization, we’re based in Cambridge, MA, and Santa Clara, CA. We’re looking for people who want to own hard problems end to end—and help shape the next generation of AI systems.
Click here to read out latest blog!
Serving frontier LLMs efficiently at scale is a moving target: every new model and workload reshuffles the tradeoffs across parallelism, batching, KV caching, and scheduling. You’ll own Netpreme’s inference-serving stack end to end, from tuning vLLM, SGLang, and TensorRT-LLM to architecting distributed serving with NVIDIA Dynamo (prefill/decode disaggregation, KV-cache-aware routing). Working directly with our hardware and systems engineers, you’ll build the benchmarks and drive the optimizations that take new models from research to production.
This role will be performed onsite from one of our offices in Santa Clara, CA or Boston, MA (relocation assistance available).
