whisper-cli produces no speaker labels on its own. Add a WhisperX/pyannote acoustic-diarization step to the zdots transcription job so its output carries real speaker turns with start/end timestamps. Define the diarized transcript schema (speaker segments + timestamps) and extend bin/validate_data + specs. These acoustic turns become the ground truth the audit skill’s heuristic speaker_map is fused onto in the next task.
Keep the existing whisper.cpp path (already Metal-accelerated at ~4x realtime per the transcription-stats retrospective, resumable, fingerprint-idempotent) as the ASR engine. Do NOT swap in WhisperX/faster-whisper as the transcriber: CTranslate2 has no real Metal/MPS backend on Apple Silicon and would run CPU int8, giving up the fast Metal path already in production.
Add a second, additive pass to the zdots transcription job:
pyannote.audio 3.x pretrained pipeline pyannote/speaker-diarization-3.1 (requires a Hugging Face token, accepted once; cache the model locally). Runs on MPS with CPU fallback.num_speakers = interviewees.length + 1 (interviewer + guests) so 2-guest interviews (e.g. Ethan Gunderson/Ryan Briones, Steve Kim/Jim Suchy) diarize to 3, not 2.-oj/SRT already carries them) plus a pyannote diarization JSON. A merge step assigns each whisper text segment the acoustic SPEAKER_xx whose turn maximally overlaps it. Output = acoustic segments with start/end/speaker/text.zdots-ctx worker --type transcription < /dev/null & (stdin redirect avoids the ffmpeg hang documented in TASK-124). Diarization is a separate fingerprinted sub-artifact so a resumed job re-uses completed whisper output and only re-runs the missing pyannote pass.Text-carrying turns stay byte-identical to today ({speaker, text} with M1/S1/S2). All timestamps live in a new top-level block, so the 196 existing structured files and the ArchiveState structured/content detection are untouched:
diarization:
engine: pyannote-3.1
model: pyannote/speaker-diarization-3.1
asr: whisper.cpp
generated_at: <iso8601>
audio_duration: 865.6
num_speakers_hint: 2
segments:
- speaker: SPEAKER_00 # acoustic label
start: 0.0
end: 12.4
text: "..." # merged whisper text (optional)
TASK-247 later adds acoustic_id: SPEAKER_00 onto each speaker_map entry to bind M1/S1/S2 → acoustic labels. Putting all timestamps here (not on turns) is the compat guarantee for 247: audit:ingest overwrites only speaker_map/turns/insights/youtube, so this block survives verbatim.
~/.config/zsh/recipes/yt-transcribe (external to this repo): add the pyannote pass + timestamp-merge; write a diarization.json sidecar next to the .txt in ~/Downloads/transcripts/<VIDEO_ID>/.bin/stage_completed_transcripts.rb: also stage the diarization.json sidecar (currently only globs *.txt) into tmp/transcript-id-staging/<video_asset_id>.diarization.json.bin/import_transcripts_from_outbox.rb (~line 393, File.write(transcript_path, { "content" => text }.to_yaml)): when a sidecar exists, write { "content" => text, "diarization" => {...} }.src/validators/site_schema.rb: add an additive TranscriptContract / diarization validator — when a diarization block is present, require engine (string), segments (array), each segment numeric start <= end and string speaker. Absent block = still valid (all 196 legacy files pass).bin/validate_data.rb (transcript loop, lines 49-54): run the new validator in addition to the existing ArchiveState parseability check.src/generators/archive_state.rb: add read-only accessors (diarization?, diarization_segments) mirroring the existing turn_texts/content_text helpers..gemini/skills/archive-forensics/SKILL.md (AC#4): update the Standard/Turbo vs max-accuracy profile note — keep whisper.cpp standard (Turbo) to avoid the loop artifacts tracked in _data/transcript_retranscribe_queue.yml; document that diarization is a fixed pyannote pass independent of the ASR profile.spec/data/transcripts_spec.rb: assert that any transcript carrying a diarization block satisfies the shape (numeric monotonic start/end, string speaker), while files without one remain valid.spec/src/generators/archive_state_spec.rb: cover the new diarization?/diarization_segments accessors on a fixture with and without the block.spec/src/validators/: add a contract spec for the additive validator (valid block, missing segments, start > end).Run one interview end-to-end through the modified job (a 2-guest one, e.g. Ethan Gunderson/Ryan Briones, to exercise the num_speakers hint); confirm the .yml gains a diarization block with sane segment timestamps covering the audio duration, turns unchanged, bin/validate_data green, and the new specs pass. bin/pipeline ci.
Depends on TASK-242 (enrichment routed onto the zdots queue). Produces the acoustic ground truth that TASK-247 fuses into the audit skill’s heuristic speaker_map; do NOT do any LLM speaker labeling here (that is 247). This task only adds the acoustic block + schema + validation.
author: claude
created: 2026-07-04 14:48
—
Ownership boundary (per target architecture): the acoustic-diarization capability (pyannote pass) is OWNED BY zdots as part of its transcription service job — that code lives in the zdots repo. This repo’s side of TASK-244 shrinks to: define the additive diarization schema block that receives zdots’ output, and the validator/spec for it. Same pattern as embedding (TASK-254): zdots computes, this repo consumes. Cross-references TASK-242 (capability boundary).
—
author: claude
created: 2026-07-05 02:14
—
CORRECTION after validating against real zdots code (scaffold: scratchpad/diarization-scaffold.md): the plan’s premise ‘pyannote is not installed / add a pyannote pass’ is WRONG. Diarization is ~80% built already — bin/diarize is a working uv/PEP-723 pyannote/speaker-diarization-3.1 script (MPS→CUDA→CPU, segment-overlap dominant-speaker merge), yt-transcribe already has a wired --diarize flag, and HUGGINGFACE_TOKEN is keychain-loaded in env.sh. ASR stays whisper.cpp/Metal (matches plan).
The REAL remaining work is wiring + output shape, in two mechanisms (one root cause = wav lifetime):
diarized stage to ingest_media.rb’s PIPELINE (the code reserves this hook; gives AC#3 resumability via pipeline_runs).<id>.speaker.json; the tenant contract needs a self-contained diarization.json (engine/model/asr/audio_duration/num_speakers_hint/segments[]). Add a --num-speakers arg — the interviewees+1 hint is SITE-side data, so it must be caller-passed. Sidecar location differs by entry point (recipe → ~/Downloads/transcripts/author: claude
created: 2026-07-05 17:00
—
zdots-side wiring landed (STAGED, uncommitted in the zdots repo, awaiting review). Three files: (1) bin/diarize — whisper_json now optional; added --sidecar (emits a self-contained diarization.json: engine/model/asr/audio_duration/num_speakers_hint/segments[]) and --num-speakers (used as a min/max bracket, not exact); legacy merge mode preserved. (2) lib/zdots/jobs/ingest_media.rb — diarized PIPELINE stage (ordered LAST, after distilled), run_diarized is opt-in via ZDOTS_DIARIZE=1 and rescue-wrapped so an unverified pyannote/torch env can’t dead-letter the ingest; diarize_audio reuses prep_audio for the retained 16k wav and writes diarization.json beside it. (3) regenerated docs/generated/ingest-pipeline.mmd.
Producer contract for the SITE consumer: sidecar is diarization.json written to the retention root ~/.local/state/zdots/ingest-sources/<mid>/<...>/ (NOT ~/Downloads) — the site’s stage_completed_transcripts.rb glob must be extended to cover it (or the job copies to Downloads). num_speakers is caller-passed via job payload num_speakers (site: interviewees + 1).
uv run must resolve a torch wheel for the mise-pinned Python 3.14.5 (flagged risk). Do one live diarize run, then flip ZDOTS_DIARIZE=1. Short-path inline --diarize (recipe + transcription.rb) remains a separate follow-up.