Sr. Software Engineer, Training/Inference (C++)
xAI · Palo Alto, CA; Seattle, WA
About this role
**SpaceXSI — Sr. Software Engineer, Training/Inference (C++)** SpaceXSI is building Super Intelligence to help humanity understand the universe. This role focuses on the high-performance inference platform that serves Grok to millions of users every day. --- **Responsibilities** - Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache) - Optimize latency and throughput of model inference under real production workloads - Build and operate high-concurrency serving systems with excellent tail latency and high reliability at massive scale - Benchmark, fine-tune, and accelerate inference engines (including low-level GPU kernel work and code generation) - Develop custom tools to trace, replay, and fix issues across the full stack (from orchestration down to GPU kernels) - Create robust CI/CD infrastructure for seamless endpoint deployment, image publishing, and inference engine updates - Accelerate research on scaling test-time compute, RL rollout, and model-hardware co-design for next-generation systems --- **Basic Qualifications** - Bachelor’s degree in computer science, computer engineering, or a science discipline - 5+ years of experience with low-level system programming in C/C++ or Rust --- **Preferred Skills and Experience** - Experience with large-scale, high-concurrent production serving - Experience with GPU inference engines (e.g., vLLM, SGLang, Triton, TensorRT-LLM) - Strong background in system optimizations: batching, caching, load balancing, parallelism - Low-level inference optimizations: GPU kernels, code generation - Algorithmic inference optimizations: quantization, speculative decoding, distillation, low-precision numerics - Experience testing, benchmarking, and ensuring reliability of inference services - Experience designing and implementing CI/CD infrastructure for inference --- **Compensation and Benefits** - **$200,000 – $375,000 USD** - Equity and comprehensive medical, vision, and dental coverage - Access to a 401(k) retirement plan - Short & long-term disability insurance, life insurance, and various other discounts/perks *SpaceXSI is an equal opportunity employer. For details on data processing, see the Recruitment Privacy Notice: https://x.ai/legal/recruitment-privacy-notice*
Listing freshness
CronJobs last confirmed this listing 2h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.