Embedded AI Engineer, On-Device Models
Deepgram · USA
About this role
## Embedded AI Engineer, On-Device Models ### Company Overview Deepgram is a leading platform powering the Voice AI economy with real-time speech-to-text (STT), text-to-speech (TTS), and production-grade voice agents. Teams worldwide build voice experiences “Powered by Deepgram,” using cloud APIs or self-hosted/on-prem software. ### About the Role Deepgram’s models run on NVIDIA GPUs—but customers need them on everything else: non-NVIDIA accelerators, embedded SoCs, mobile processors, DSPs/NPUs, and purpose-built devices with tight memory, compute, thermal, and power budgets. As an **Embedded AI Engineer** on the **Partner Platform Engineering** team, you’ll work at the lowest layer of the edge stack: - Write and optimize **custom kernels and operators** for specific hardware - Collapse models onto device execution units - Perform **target-side quantization** and **assembly-level tuning** beyond standard toolchains - Deliver reusable runtime components that Applied ML Engineers can use to deploy models on new silicon ### What You’ll Do - Write and optimize custom kernels/operators (C, C++, Rust, and platform assembly/intrinsics) for non-NVIDIA accelerators, embedded SoCs, DSPs, and NPUs - Own target-side optimization to meet **latency, memory, power, and thermal** budgets (quantization, operator fusion, memory layout, architecture-specific compilation) - Integrate with vendor NPU/DSP toolchains and edge inference runtimes; extend them with custom operators when needed - Build performance-critical runtime code for embedded Linux, bare-metal, and RTOS targets - Establish per-platform benchmarking/validation (latency, accuracy, power, memory footprint, utilization) and prevent regressions - Partner with silicon/platform vendors on SDK integration and low-level performance tuning - Feed hardware constraints back to Applied ML and Research to improve model designs for constrained targets ### You’ll Love This Role If - You enjoy making large models run on hardware that wasn’t designed for them—while still hitting accuracy and latency targets - You prefer profiling, ISA manuals, and “close to the metal” problem solving over scaling up hardware - You care about real-world details like cold start, power draw, thermals, memory fragmentation, and cache behavior - You like hard, constrained, ship-it engineering problems ### It’s Important To Us That You Have - Experience delivering production systems on **resource-constrained hardware** (embedded/mobile/edge AI/small low-power devices) - Strong proficiency in **C, C++, and/or Rust**; performance-critical code for constrained environments - Hands-on model optimization for on-device deployment (e.g., quantization, pruning, distillation, architecture-specific compilation) - Familiarity with edge inference runtimes (ONNX Runtime, TensorRT, TFLite, ExecuTorch) and/or vendor NPU/DSP toolchains - Strong understanding of hardware-software interaction (CPU/GPU/NPU/DSP, memory hierarchies, fixed-point/integer arithmetic, power management) - Experience working close to the metal (bare-metal/RTOS like FreeRTOS/Zephyr, embedded Linux, microcontrollers/edge SoCs) - Strong communication skills and a builder mindset (scope ambiguous optimizations, measure results, explain tradeoffs) ### It Would Be Great If You Had - Real-time audio processing experience on embedded platforms (DSP pipelines, codec optimization, wake-word/always-on, streaming inference) - Depth in ML optimization for edge targets (custom qua
Listing freshness
CronJobs last confirmed this listing 2h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.