Embedded AI Engineer, On-Device Models
Deepgram · USA - Remote
About this role
## Embedded AI Engineer — On-Device Models ### Company Overview Deepgram is the leading platform underpinning the Voice AI economy, offering real-time speech-to-text (STT), text-to-speech (TTS), and production-grade voice agents. With voice-native foundation models available via cloud APIs or self-hosted/on-prem software, Deepgram powers voice offerings for 1,300+ organizations and 200,000+ developers. ### About the Role As an **Embedded AI Engineer** on the **Partner Platform Engineering** team, you’ll work at the lowest layer of Deepgram’s edge stack. You’ll write and optimize **custom kernels and operators** for specific hardware, collapse models onto device execution units, and perform **target-side quantization and assembly-level tuning** when standard toolchains can’t run Deepgram models well. Your work makes new hardware platforms viable for Deepgram. ### What You’ll Do - Write and optimize **custom kernels and operators** (C, C++, Rust, and platform assembly/intrinsics) for non-NVIDIA accelerators, embedded SoCs, DSPs, and NPUs. - Own **target-side optimization**: quantization, operator fusion, memory layout, and architecture-specific compilation to meet **latency, memory, power, and thermal** budgets. - Integrate with **vendor NPU/DSP toolchains** and edge inference runtimes; extend them with custom operators when graphs don’t map cleanly. - Deliver reusable **kernel/runtime building blocks** with clear interfaces and documented constraints. - Build performance-critical runtime code for **embedded Linux, bare-metal, and RTOS** targets. - Establish **per-platform benchmarking and validation** (latency, accuracy, power, memory footprint, utilization) and catch regressions before release. - Partner with **silicon and platform vendors** on SDK integration and low-level performance tuning for new chipsets. - Feed hardware constraints back to **Applied ML** and **Research** to help model designs land on constrained targets. ### You’ll Love This Role If - You enjoy making large models run on hardware not originally meant for them—while still hitting accuracy and latency targets. - You reach for the profiler and ISA manual before asking for a bigger chip. - You’d rather write the kernel than wait for the vendor. - You care about details beyond cloud benchmarks (cold start, power draw, thermals, memory fragmentation, cache behavior). - You prefer hard, constrained, ship-it problems over open-ended research. ### It’s Important That You Have - Experience delivering **production systems** on **resource-constrained hardware** (embedded/mobile/edge AI/small low-power devices). - Strong proficiency in **C, C++, and/or Rust**, including performance-critical code for constrained environments. - Hands-on experience with **model optimization for on-device deployment** (quantization, pruning, knowledge distillation, architecture-specific compilation). - Familiarity with edge inference runtimes (e.g., **ONNX Runtime, TensorRT, TFLite, ExecuTorch**) and/or vendor-specific NPU/DSP toolchains. - Strong understanding of **hardware-software interaction** (CPU/GPU/NPU/DSP architectures, memory hierarchies, fixed-point/integer arithmetic, power management). - Experience working close to the metal: **bare-metal/RTOS** (e.g., FreeRTOS, Zephyr), embedded Linux, or microcontroller/edge SoC development. - Strong communication skills and a builder mindset (scope ambiguous problems, drive measurable results, explain tradeoffs). ### It Would Be Great If Yo
Listing freshness
CronJobs last confirmed this listing 1h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.