@hiyve/react-music-performance

React hooks and components for practising from a score, on @hiyve/music-performance: record a take, follow the performer on the sheet music as they play, grade the take, keep it, and write sight-reading exercises.

React hooks and a component for recording a take — unprocessed microphone audio plus MIDI from a connected instrument, on one clock — so a performance can be saved and analysed against a score later.

Quick Start

import { TakeRecorder } from '@hiyve/react-music-performance';

function Practice() {
  return (
    <TakeRecorder
      score={{ name: 'Minuet in G' }}
      onTakeCompleted={(bundle) => save(bundle)}
      onError={(error) => console.error(error)}
    />
  );
}
Copy

Press Record, play, press Stop. The finished take arrives in onTakeCompleted as a TakeBundle — a JSON record plus a WAV blob — and the component offers both as downloads.

Custom UI with hooks

import { useTakeRecorder } from '@hiyve/react-music-performance';

function CustomRecorder() {
  const recorder = useTakeRecorder({ profile: 'instrument', onTakeCompleted: save });

  return (
    <button onClick={recorder.isRecording ? recorder.stop : recorder.start}>
      {recorder.isRecording ? 'Stop' : 'Record'}
    </button>
  );
}
Copy

Grading a take against a score

import { gradeTake } from '@hiyve/music-performance';
import { TakeRecorder, TakeReport, useScore } from '@hiyve/react-music-performance';

function Practice() {
  const { score, takeScore, loadFile, error } = useScore();
  const [report, setReport] = useState(null);

  return (
    <>
      <input type="file" accept=".musicxml,.xml" onChange={(e) => e.target.files?.[0] && void loadFile(e.target.files[0])} />
      {error && <p>{error.message}</p>}
      <TakeRecorder
        score={takeScore}
        onTakeCompleted={(bundle) => score && setReport(gradeTake(score, bundle.take))}
      />
      {report && <TakeReport report={report} />}
    </>
  );
}
Copy

useScore reads a .musicxml file (compressed .mxl is not supported yet), gradeTake compares the take's MIDI with the score, and TakeReport shows the result: accuracy, counts of correct / wrong / missing / extra notes, the tempo the performer held against the score's, and a per-note table with early/late timing.

Live feedback while playing

import { MusicXmlPlayer } from '@hiyve/react-music-notation';
import { TakeRecorder, TakeReport, liveNoteStyles, useLivePerformance, useScore } from '@hiyve/react-music-performance';

const loadEngine = () => import('opensheetmusicdisplay');

function Practice({ scoreUrl }: { scoreUrl: string }) {
  const { score, takeScore } = useScore();
  const live = useLivePerformance({ score, source: 'audio' }); // 'midi' for a MIDI instrument

  return (
    <>
      <TakeRecorder
        score={takeScore}
        onRecordingStarted={live.reset}
        onAudioChunk={live.pushAudio}
        onMidiEvent={live.pushMidi}
        onTakeCompleted={() => live.finish()}
      />
      <MusicXmlPlayer
        variant="inline"
        fileUrl={scoreUrl}
        loadEngine={loadEngine}
        noteStyles={liveNoteStyles(live.snapshot)}
        position={live.position}
      />
      {live.report && <TakeReport report={live.report} />}
    </>
  );
}
Copy

useLivePerformance runs a LiveSession (from @hiyve/music-performance) on the audio — or the MIDI — as it is captured. Playing freely, every note is compared with the next one written and the session moves on, right or wrong; each expected note gets a verdict in snapshot.verdicts, keyed by score-note id: a quick provisional one about 150 ms after the attack, then a confirmed one about 300 ms after. snapshot.cursor and snapshot.next say where the performer is, snapshot.tempoBpm what tempo they are holding, snapshot.extras what was heard that the score does not explain. finish() grades the whole session as a PerformanceReport for TakeReport.

By default the session follows the performer's own tempo. To judge against a set one — a metronome's — add useMetronome and tell the live hook where the first beat falls and what the click sounds like, so a click the microphone hears through speakers is found during the count-in and taken out of the audio before it is read (snapshot.clickOffsetMs says how late the clicks arrive, and the beat is judged where they were heard; through headphones none is found and nothing is touched):

const metronome = useMetronome({ bpm: 100, beatsPerBar: 4, countInBars: 1 });
const live = useLivePerformance({ score });

<TakeRecorder
  onRecordingStarted={({ nowMs }) => {
    live.reset();
    const started = metronome.start(); // a one-bar count-in, then the piece
    if (started) {
      live.setTempo({
        bpm: 100,
        startTakeMs: nowMs() + (started.firstBeatPerformanceMs - performance.now()),
        nowMs,
        click: { ...started.click, countInBeats: started.countInBeats },
      });
    }
  }}
  onAudioChunk={live.pushAudio}
  onTakeCompleted={() => {
    metronome.stop();
    live.finish();
  }}
/>
Copy

Timing is then early or late against the beat, a note whose beat passes unplayed turns orange at once (and green if it arrives late), and the position line glides at the beat. How closely a note must sit on its beat is the strictness — 'relaxed', 'normal' (the default) or 'strict', windows of 150, 80 and 40 ms (timingStrictness): inside the window a note is simply on time; outside it the score shows where it was played, and the longer window also means a beat has to pass by more before an unplayed note is called missing. Without a set tempo timing is not judged at all — a note is right or wrong by its pitch. The clicks are noise rather than tones, so a microphone that hears them does not take them for notes — but headphones keep them out of the recording altogether.

On the score itself, liveNoteStyles(snapshot, options?) turns the verdicts into the noteStyles the MusicXmlPlayer from @hiyve/react-music-notation colours its note heads by — green, red and orange by default (the TakeReport colours), fainter while a verdict is still provisional — and, under a set tempo, gives every note played outside the strictness window (strictness, or an explicit markTimingAfterMs) a mark where it was actually played, beside the engraved head. live.position is the player's position line: on the note expected next while following the performer, and under a set tempo a line gliding at the beat, which the player animates between renders (livePositionQuarters(snapshot) gives the plain note position for other views). The same score file must be given to both: useScore for the engine that follows, and fileUrl for the engine that draws.

A practice page in one hook

usePracticeSession is recording, following and grading in one. Give it the score and how the take is paced, hand its styles and position to the score player, put the toolbar controls beside the player's own, and the take ends when the piece does — graded as surely as it can be: MIDI, then the audio, then real time.

import { useRef, useState } from 'react';
import { MusicXmlPlayer } from '@hiyve/react-music-notation';
import type { MusicXmlPlayerHandle } from '@hiyve/react-music-notation';
import {
  LiveStrip,
  PacingSwitch,
  PracticeSettingsButton,
  RecordButton,
  TakeScoreCard,
  defaultPracticeSettings,
  practiceRecorderOptions,
  takeResultView,
  useAudioInputs,
  useMidiInputs,
  usePracticeSession,
  useScore,
  useScoreUrl,
  useTakeHistory,
} from '@hiyve/react-music-performance';
import type { PacingMode } from '@hiyve/react-music-performance';

function PracticePage({ loadEngine }: { loadEngine: NotationEngineLoader }) {
  const { score, name, musicxml } = useScore();
  const url = useScoreUrl(musicxml);
  const playerRef = useRef<MusicXmlPlayerHandle>(null);
  const [mode, setMode] = useState<PacingMode>('free');
  const [settings, setSettings] = useState(defaultPracticeSettings);
  const history = useTakeHistory();
  const midiInputs = useMidiInputs();
  const audioInputs = useAudioInputs();
  const session = usePracticeSession({
    score,
    musicxml,
    title: name,
    mode,
    bpm: 100,
    strictness: settings.strictness,
    instrument: settings.instrument,
    liveFromMidi: settings.liveFromMidi,
    ...practiceRecorderOptions(settings),
    playerRef,
    onTakeGraded: history.save,
  });
  return (
    <>
      {url && (
        <MusicXmlPlayer
          ref={playerRef}
          fileUrl={url}
          loadEngine={loadEngine}
          noteStyles={session.noteStyles}
          position={session.position}
          transportActions={<RecordButton recorder={session.recorder} />}
          toolbarActions={
            <>
              <PacingSwitch mode={mode} onChange={setMode} disabled={session.recorder.isRecording} />
              <PracticeSettingsButton
                settings={settings}
                onChange={(next) => setSettings((current) => ({ ...current, ...next }))}
                recorder={session.recorder}
                midiInputs={midiInputs}
                audioInputs={audioInputs}
                paced={mode !== 'free'}
              />
            </>
          }
          onPlaybackEnded={session.handlePlaybackEnded}
        />
      )}
      {session.live.snapshot && <LiveStrip snapshot={session.live.snapshot} expectClicks={mode === 'metronome'} note={session.takeNote} />}
      {session.latest && <TakeScoreCard view={takeResultView(session.latest)} saved />}
    </>
  );
}
Copy

The instrument chosen in the settings decides how the microphone is listened to (instrumentProfiles: keyboard, sustained, voice). A take ends on its own at ten minutes, and a free take that has heard nothing for two minutes ends too; maxTakeMs and idleEndMs change that.

Sight-reading exercises

SightReadingPanel is the settings of an exercise as a form; the page writes the tune with generateSightReading and loads it like any score. Every exercise has a code — its settings and seed in sixteen characters — shown in the panel and, with ExerciseCodeChip, beside the score. A take of the exercise keeps the code when the session is told which exercise is on the card, so a saved take can be tried again from its code later.

const [settings, setSettings] = useState(defaultSightReadingSettings);
const exercise = useMemo(() => generateSightReading({ ...settings, tempoBpm: 80 }), [settings]);
useEffect(() => {
  loadText(exercise.musicxml, exercise.title);
}, [exercise, loadText]);
const session = usePracticeSession({ …, exercise: { code: exercise.code, musicxml: exercise.musicxml } });

<SightReadingPanel settings={settings} onChange={setSettings} locked={session.recorder.isRecording} />
<ExerciseCodeChip code={exercise.code} />
Copy

To try a saved take again, parseSightReadingCode(row.exercise) gives the settings back; a take saved before exercises had codes can still be found from its title and stored tune with recoverSightReadingSettings.

Results and history

useTakeHistory keeps graded takes in this browser — the numbers in a list, the marked score alongside. TakeScoreCard shows a take's score, MarkedScore its score marked note by note, and SavedTakesTable the takes kept, each with Try again.

const history = useTakeHistory();
const [picked, setPicked] = useState<{ row: SavedTakeResult; stored?: StoredTakeScore } | null>(null);
const view = picked ? savedTakeResultView(picked.row, picked.stored) : session.latest ? takeResultView(session.latest) : null;

{view && <TakeScoreCard view={view} saved={picked !== null} onTryAgain={() => tryAgain(view)} />}
{view?.musicxml && <MarkedScore view={view} loadEngine={loadEngine} />}
<SavedTakesTable
  rows={history.results}
  selectedId={picked?.row.id ?? null}
  onOpen={(row) => void history.getStoredScore(row.id).then((stored) => setPicked({ row, stored }))}
  onTryAgain={(target) => tryAgain(target)}
  onClear={history.clear}
/>
Copy

Saving takes to a room's file storage — the /room entry

import { TakeRecorder } from '@hiyve/react-music-performance';
import { useSaveTake } from '@hiyve/react-music-performance/room';

function Practice() {
  const { saveTake, isSaving, lastSaved } = useSaveTake({ location: '/Takes' });

  return (
    <>
      <TakeRecorder onTakeCompleted={(bundle) => void saveTake(bundle)} />
      {isSaving && <p>Saving…</p>}
      {lastSaved && <p>Saved as {lastSaved.takeFileId}</p>}
    </>
  );
}
Copy

useSaveTake uploads the WAV and the JSON record to the performer's file storage and links the two through their metadata, so either file leads to the other. It needs <HiyveProvider> with a signed-in user; no room connection is required. It is the one part of the package that needs @hiyve/react, and lives behind the /room entry so everything else works without it.

Try it

pnpm --filter @hiyve/react-music-performance playground
Copy

Opens a page with the recorder wired to your real MIDI keyboard and microphone, and the score drawn by @hiyve/react-music-notation's player. With the public opensheetmusicdisplay build the player renders and steps; to have its playback, tempo slider and per-instrument mixer as well — and to pace a take with the score's own playback — drop the engine build with audio into packages/react-music-performance/dev/public/vendor/opensheetmusicdisplay.min.js. That folder is git-ignored: the build is licensed to the app, not the SDK, and is never committed. The playground says which engine it found.

Features

• Picks a MIDI input (one, all, or none), a microphone, and a capture profile
• Records audio unprocessed by default — no echo cancellation, noise suppression, or automatic gain
• Shows which processing flags the browser actually honoured, and warns when it kept some on
• Live level meter, elapsed time, and MIDI event counter
• Measures the take's own MIDI-to-audio offset from the notes played and stores it as take.alignment (shown after each take)
• Hands audio chunks and MIDI events out as they arrive, so useLivePerformance can follow the score while recording
• Downloads the take as WAV and JSON
• Releases the microphone and MIDI ports if unmounted mid-take
• usePracticeSession: a take that ends when the piece does, graded as surely as it can be, with the controls, the live readout and the score card to go with it
• Sight-reading exercises with a code that makes the same tune again anywhere
• Takes kept in the browser with their marked scores, and tried again from the list

Components

Hooks

TakeReport Props

Utilities

usePracticeSession Options

Return value

useTakeHistory Options

Returns results (newest first), save(take), update(id, patch), clear() and getStoredScore(id). Nothing leaves the browser.

TakeScoreCard Props

ScoreRing takes value (0–1 or null), size (150), label ("score") and the good / fair / poor colours; ExerciseCodeChip takes code, sx and labels (copy, copied).

MarkedScore Props

SavedTakesTable Props

SightReadingPanel Props

LiveStrip Props

PacingSwitch, RecordButton and PracticeSettingsButton Props

A PracticeSettings holds profile, midiSelection ('all', 'none' or an input id), audioSelection ('default' or a device id), liveFromMidi, strictness and instrument; defaultPracticeSettings is the place to start and practiceRecorderOptions turns one into recorder options.

useSaveTake Options (/room entry)

The stored JSON record carries metadata (takeId, audioFileId, scoreName, durationMs, noteOnCount, midiToAudioOffsetMs, score, gradeSource) and the WAV carries takeId and takeFileId. saveTake(bundle, { grade: takeGradeOf(take) }) writes the take's grade into the record itself, so the file says how the take went.

TakeRecorder Props

useTakeRecorder Options

Return value

useLivePerformance Options

Return value

A session starts with the first input after a score is set and after each finish() or reset(), so one hook serves take after take; changing the score, region or source discards the current one.

useMetronome Options

Return value

Customization

Every visible string is a label; pass a partial object to override any of them.

<TakeRecorder
  labels={{ record: 'Start', stop: 'Finish', midiEventCount: '{count} notes' }}
  colors={{ recording: '#c62828', levelMeter: '#2e7d32' }}
  icons={{ record: <MyIcon /> }}
/>
Copy

Requirements

• React 18 or 19
• @mui/material and @mui/icons-material v9 with Emotion
• @hiyve/music-performance 0.1 or later and @hiyve/react-music-notation 0.2 or later (with a notation engine of your own); @hiyve/utilities and @hiyve/rtc-client
• A browser with getUserMedia and AudioWorklet; Web MIDI for MIDI capture (not available in Safari)
• TakeRecorder, useTakeRecorder, useMidiInputs and useAudioInputs work without HiyveProvider; the component opens its own microphone track, so use it when no call is in progress.
• useSaveTake, from the /room entry, must be inside <HiyveProvider> with a signed-in user. @hiyve/react is needed for that entry only.
• MarkedScore draws the score with @hiyve/react-music-notation's player, so it needs a notation engine through loadEngine, as the player does.