Google’s New On-Device Dictation App ‘Eloquent’ Fails to Transcribe Half of Speech, Benchmarking Attempt Shows
English summary
A user attempted to benchmark Google's new Eloquent local dictation app but found that it dropped about half of dictations, returning only a small fraction of spoken words. In 15 of 50 tests, the app provided a complete transcript with a word error rate of ~24%, comparable to Qwen3-ASR's ~21%. However, for the majority of attempts, the output was severely incomplete, with clips of 20+ words often yielding just 5-10 words. The user suspects the underlying chat-style AI model sometimes refuses to transcribe and instead responds with an apology, a behavior observed when running Gemma 3n directly on the same audio. The issue highlights a fundamental usability problem with the chat-based transcription approach.
Chinese summary
一名用户尝试对谷歌新发布的设备端听写应用Eloquent进行基准测试，发现约一半的听写内容会被丢弃，返回的文本严重缺失。在50次测试中，仅15次获得完整转录，词错误率约24%，与Qwen3-ASR的21%相近。多数情况下，一段20多词的音频仅转录出5-10个词。用户怀疑底层的对话式AI模型有时会拒绝转录并回复道歉，这一现象在直接使用Gemma 3n模型时同样出现，表明基于对话模型的听写方式存在根本性可用性问题。
Key points
Eloquent dropped roughly 50% of dictations, returning only partial text (often 5–10 words from 20+ word clips).
Eloquent在约一半的听写中丢失大量内容，20多词的音频常仅返回5-10个词。
When it worked, Eloquent achieved ~24% WER vs. Qwen3-ASR’s ~21%, but the frequent failures make the app unusable.
当能完整转录时，Eloquent的词错误率约为24%，与Qwen3-ASR的21%相近，但频繁的缺失导致其不可用。
The same refusal behavior was observed with Gemma 3n, suggesting a chat-model design flaw rather than a bug in the app's pipeline.
直接在Gemma 3n上测试音频时同样出现拒绝转录的回复，表明这是对话模型设计缺陷，而非应用流水线错误。