Jargon Bench scorecard

How well Deepgram and ElevenLabs speech-to-text hear physical-security vocabulary, with and without vocabulary boosting. Runs on 2026-10-04 (UTC); list prices as of 2026-10-03.

Headline: jargon accuracy on recorded speech

44 clips from one speaker, each sent 3 times per provider and condition (58 term occurrences per pass).

Deepgram nova-3

62.1%→89.7%

108 of 174 → 156 of 174 jargon terms, baseline → boosted

ElevenLabs scribe_v2

79.3%→95.4%

138 of 174 → 166 of 174 jargon terms, baseline → boosted

Recommendation

Turn on keyterm boosting for field audio. On recorded speech it raised jargon accuracy from 62.1% to 89.7% with Deepgram and from 79.3% to 95.4% with ElevenLabs.

The cost was small: median latency changed by 9 ms and 21 ms, and list price rose from $0.26 to $0.34 and from $0.22 to $0.27 per audio hour.

On this dataset ElevenLabs was more accurate (95.4% vs 89.7% boosted) and Deepgram answered faster (median 128 ms vs 472 ms).

Review the boost list for terms that sound like everyday words: boosting wrote "Axis" and "supervisory" where it wasn't said (5 clip-level cases across all audio).

All results by audio source

Sources are never averaged together. TTS audio was generated with ElevenLabs voices, so it can favor ElevenLabs speech-to-text.

Recorded speech

Provider	Condition	Jargon terms	WER	Median	p95	Failed	False positives	Clips × repeats
Deepgram	baseline	62.1% (108 of 174)	11.7%	119 ms	598 ms	0 of 132	0	44 × 3
Deepgram	boosted	89.7% (156 of 174)	8.6%	128 ms	597 ms	0 of 132	3	44 × 3
ElevenLabs	baseline	79.3% (138 of 174)	5.7%	451 ms	711 ms	0 of 132	0	44 × 3
ElevenLabs	boosted	95.4% (166 of 174)	3.5%	472 ms	703 ms	0 of 132	3	44 × 3

Recorded speech + noise (10 dB SNR)

Provider	Condition	Jargon terms	WER	Median	p95	Failed	False positives	Clips × repeats
Deepgram	baseline	46.6% (27 of 58)	17.7%	180 ms	434 ms	0 of 44	0	44 × 1
Deepgram	boosted	86.2% (50 of 58)	13.7%	178 ms	868 ms	0 of 44	1	44 × 1
ElevenLabs	baseline	75.9% (44 of 58)	9.1%	515 ms	933 ms	0 of 44	0	44 × 1
ElevenLabs	boosted	96.6% (56 of 58)	5.5%	541 ms	701 ms	0 of 44	2	44 × 1

TTS (ElevenLabs-generated)

Provider	Condition	Jargon terms	WER	Median	p95	Failed	Clips × repeats
Deepgram	baseline	83.1% (49 of 59)	3.0%	314 ms	580 ms	0 of 45	45 × 1
Deepgram	boosted	98.3% (58 of 59)	0.7%	313 ms	662 ms	0 of 45	45 × 1
ElevenLabs	baseline	96.6% (57 of 59)	0.7%	477 ms	672 ms	0 of 45	45 × 1
ElevenLabs	boosted	100.0% (59 of 59)	0.4%	473 ms	783 ms	0 of 45	45 × 1

Every term, recorded speech

Occurrences heard correctly across all repeats. "Heard as" is the most common baseline mistake.

Term	Deepgram baseline	Deepgram boosted	ElevenLabs baseline	ElevenLabs boosted	Heard as (baseline)
NVR	0/9	9/9	9/9	9/9	Deepgram: “mbr”
VMS	3/6	6/6	6/6	6/6	Deepgram: “bms”
PTZ	3/3	3/3	3/3	3/3
WDR	0/3	3/3	3/3	3/3	Deepgram: “wdor”
fisheye	3/3	3/3	3/3	3/3
varifocal	0/3	3/3	3/3	3/3	Deepgram: “rear focal”
ONVIF	0/3	3/3	3/3	3/3	Deepgram: “on vif”
REX	6/6	6/6	6/6	6/6
mag lock	6/6	6/6	6/6	6/6
fail safe	0/3	0/3	0/3	3/3	Deepgram: “failed safe” ElevenLabs: “and failed safe”
fail secure	3/3	3/3	3/3	3/3
electric strike	0/3	3/3	0/3	3/3	Deepgram: “electric stripe” ElevenLabs: “electric stripe”
Wiegand	0/3	0/3	0/3	3/3	Deepgram: “the weekend” ElevenLabs: “weekend”
OSDP	3/3	3/3	3/3	3/3
door contact	6/6	6/6	6/6	6/6
card reader	3/3	3/3	3/3	3/3
glass break	0/3	0/3	0/3	0/3	Deepgram: “glass” ElevenLabs: “glass”
PIR	3/3	3/3	3/3	3/3
PoE	6/9	6/9	6/9	7/9	Deepgram: “the puh” ElevenLabs: “the peh”
VLAN	3/3	3/3	3/3	3/3
IDF	3/6	6/6	6/6	6/6	Deepgram: “ivf”
MDF	3/3	3/3	3/3	3/3
Cat6	6/6	6/6	6/6	6/6
SFP	3/3	3/3	3/3	3/3
J-hook	3/3	3/3	3/3	3/3
conduit	3/3	3/3	3/3	3/3
Cisco Catalyst	3/3	3/3	3/3	3/3
Meraki	3/3	3/3	3/3	3/3
Ubiquiti	3/3	3/3	3/3	3/3
Verkada	0/3	3/3	3/3	3/3	Deepgram: “vricata”
Axis	0/3	0/3	0/3	3/3	Deepgram: “access” ElevenLabs: “access”
Hanwha	0/3	3/3	0/3	3/3	Deepgram: “hot wall” ElevenLabs: “hall one”
Genetec	0/3	3/3	0/3	3/3	Deepgram: “genitex” ElevenLabs: “genetech”
Milestone	3/3	3/3	3/3	3/3
Avigilon	0/6	6/6	3/6	6/6	Deepgram: “a digilon” ElevenLabs: “uh vigilant”
Mercury	6/6	6/6	6/6	6/6
DMP	6/6	6/6	6/6	6/6
Lenel	0/3	3/3	0/3	3/3	Deepgram: “lonell” ElevenLabs: “linnell”
Brivo	0/3	3/3	0/3	3/3	Deepgram: “maribo” ElevenLabs: “horiba”
Openpath	3/3	3/3	3/3	3/3
Aiphone	0/3	0/3	0/3	0/3	Deepgram: “ai” ElevenLabs: “ai”
central station	6/6	6/6	6/6	6/6
tamper	3/3	3/3	3/3	3/3
supervisory	3/3	3/3	3/3	3/3

False positives

A term written where it wasn't said. Jargon accuracy can't see these, so they're counted separately.

Audio	Provider	Clip	Term	Repeats	Transcript
Recorded speech	Deepgram boosted	u036	“Axis”	3	We've quoted Avigilon for cameras and Mercury hardware for the Axis Control Panel.
Recorded speech	ElevenLabs boosted	u013	“supervisory”	3	The readers are still on Wiegand, so we should plan to move to OSDP for supervisory
Recorded speech + noise (10 dB SNR)	Deepgram boosted	u023	“Axis”	1	The Meraki dashboard shows the switchboard for that Axis.
Recorded speech + noise (10 dB SNR)	ElevenLabs boosted	u013	“supervisory”	1	The readers are still on Wiegand, so we should plan to move to OSDP for supervisory
Recorded speech + noise (10 dB SNR)	ElevenLabs boosted	u023	“Axis”	1	The Meraki dashboard shows the switch port for that Axis board flattening all night

Caveats

44 recorded clips from one speaker, each sent 3 times; 58 jargon-term occurrences.
Noise is steady synthetic pink noise; real job sites are usually harder. TTS audio came from ElevenLabs voices and is excluded from the headline.
Batch transcription only. Models and list prices as of 2026-10-03; runs on 2026-10-04 (UTC).

Built from 2026-10-04T02-20-47-223Z, 2026-10-04T02-40-00-605Z, 2026-10-04T02-52-00-161Z, 2026-10-04T02-53-18-454Z. Dataset: 45 utterances, 44 terms. Method, code, and raw-result format: github.com/mrkygray-art/nightagent/tree/main/bench.