Design and Evaluation of Low-Latency Voice Assistant Systems with Memory–Thread Co-Optimization
PDF

Keywords

voice assistants
low latency
system stability
memory control
multithread scheduling
edge devices
speech processing

Abstract

Intelligent voice assistants must keep delay low and service steady on devices with limited compute  and power. We  applied  a  dual-path  design  that  combines  fine memory  control with multithread scheduling. A total of 126 trials were run on a smartphone SoC, an embedded Linux board, and a wearable-class microcontroller, in both booth and office settings. Power on the main rail was sampled and aligned with software logs. Idle energy was measured and subtracted. The design cut median latency by about 37% and 95th-percentile latency by about 40%. Jitter fell by roughly  25%,  and  timeout  events   dropped  by  74%  versus  the   standard  setup.  Throughput increased by 23–28%. Memory pools and pinned buffers shortened allocation time by 63% and reduced DRAM traffic by 34%, improving the energy–delay product by  18–24%. Recognition accuracy stayed stable (WER change ≤0.2; F1 change ≤0.3). In sum, combining memory control and task priority reduced delay and improved stability without loss of accuracy. The design suits phones, wearables, and embedded boards that require fast and steady response. Limits include a small set of devices, short test time, and English-only data. Future work will add more hardware types, longer trials, and secure execution tests.

PDF
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright (c) 2025 Samuel J. Brooks, Olivia R. Bennett, Yuting Li, Karen H. Ng (Author)

Downloads

Download data is not yet available.