Better Office Hours

A voice tutor that teaches on a live whiteboard and the student's own notes, and gives hints instead of answers.

I built the voice loop, the whiteboard engine, and the workspace · Sep to Oct 2026

Recognition: 1st place · YaleAI x SpaceXAI Cursor Build Challenge

Results

• 1st place, YaleAI x SpaceXAI Cursor Build Challenge
• 2 days to the winning prototype

The problem

Office hours are crowded and intimidating, and generic AI tutors tend to solve the problem instead of teaching it. A 2025 PNAS field study found unguarded AI help raised practice scores but cut later unassisted exam performance by 17 percent.

What I built

• A custom SVG whiteboard engine: progressive handwriting reveal, LaTeX typeset locally, generated animations with play, pause and scrub, and separate tutor and student ink layers.
• The voice loop: on-device speech detection, ElevenLabs transcription and speech, and tap-to-interrupt that stops speech and freezes the animation without losing the board.
• A PDF desk where the tutor points and highlights, and sees page and board snapshots that include the student's own marks.
• Two model lanes: a fast model acknowledges and routes, and a reasoning model returns a teaching move plus up to three narrated drawing beats, with guards that hold back answers on graded work.

The hard part

Keeping generated drawings in step with a voice that is synthesized, buffered, and interruptible. Model output streams faster than speech plays, so drawing immediately would show the picture (or the answer) before the tutor said it. I queued each visual command behind the words before it and released it only when that audio actually started playing. A playback epoch means an interruption cancels every pending visual, and late results are ignored.

How it fits together

• Mic + speech detection to Transcription (utterance)
• Transcription to Agent route (transcript)
• Agent route to Fast lane
• Fast lane to Speech-synced queue (quick reply)
• Agent route to Reasoning lane (lesson)
• Reasoning lane to Speech-synced queue (speech + visuals)
• Speech-synced queue to Whiteboard + PDF desk (on playback)

Invariant: nothing appears on the board before the tutor says it.

Decisions

• Drawings are released by the audio's own playing event, not by timers, because synthesized speech starts and buffers unpredictably.
• Two model lanes: a fast model acknowledges right away while a reasoning model plans the lesson, so the tutor never sits silent.
• Math is typeset locally with a pinned MathJax build instead of a CDN, so the board renders the same way every time.

Tech: Next.js, TypeScript, ElevenLabs, Voice AI. Code

By David Antwi (david.antwi@yale.edu). Markdown version: /work/better-office-hours.md. Full profile: /llms.txt.