Getting started with spessasynth_core

spessasynth_core vs spessasynth_lib

There are two similar libraries: spessasynth_lib and spessasynth_core:

• spessasynth_core is the main library that contains all the MIDI, SF2 and DLS parsing and the synthesis engine. It can run in any JS environment.
• spessasynth_lib builds on top of spessasynth_core, relying on WebAudioAPI and WebMIDIAPI to add easy-to-use wrappers for the SpessaSynthProcessor and SpessaSynthSequencer.

So:

Use spessasynth_lib if:

• You want to play MIDI files in the browser without much work.
• You don't want to have to program your own audio processor.
• The default effects are good enough for you.
• You don't need direct access to the audio engine.

Use spessasynth_core if:

• You want access to raw PCM samples.
• You want custom effect processors.
• You need full control over the audio.
• You don't need spessasynth_lib wrappers.
• You don't have access to the WebAudioAPI.

Installation

npm install --save spessasynth_core
Copy

Minimal Setup

A minimal setup for the synthesizer involves parsing a sound bank and adding it to a new synthesizer:

import { SoundBankLoader, SpessaSynthProcessor } from "spessasynth_core";

// `soundBankBuffer` is an ArrayBuffer containing an SF2, SF3 or DLS file
const soundBank = SoundBankLoader.fromArrayBuffer(soundBankBuffer);
const synth = new SpessaSynthProcessor(48_000); // Sample rate in Hertz
synth.soundBankManager.addSoundBank(soundBank, "main");
await synth.ready;
Copy

Understanding the audio loop

spessasynth_core provides very raw access to the audio data, outputting float PCM samples. These samples can then be sent to speakers, saved somewhere or processed, for example in an AudioWorklet's process method.

Example MIDI player audio loop

Here is the most basic audio loop for the synthesizer and sequencer, continuing from the minimal setup above:

import { BasicMIDI, SpessaSynthSequencer } from "spessasynth_core";

// `midiBuffer` is an ArrayBuffer containing a MIDI file
const sequencer = new SpessaSynthSequencer(synth);
sequencer.loadNewSongList([BasicMIDI.fromArrayBuffer(midiBuffer)]);
sequencer.play();

const bufferSize = 128;
// Allocate the buffers once and reuse them
const samplesL = new Float32Array(bufferSize);
const samplesR = new Float32Array(bufferSize);
while (true) {
    sequencer.processTick();
    // The synthesizer mixes into the buffers, so clear them first
    samplesL.fill(0);
    samplesR.fill(0);
    synth.process(samplesL, samplesR);
    // Send the audio somewhere here
}
Copy

This loop processes the sequencer before rendering the audio to the buffers.

Also keep in mind that a single process call cannot render more than maxBufferSize samples (128 by default, larger values throw an error), as the process function calculates the envelopes and LFOs once, so buffer size represents the shortest amount of time between those changes.

To fill a larger buffer, render it in chunks:

// Divisible by 128
const outL = new Float32Array(2048);
const outR = new Float32Array(2048);
// 2048 / 128 = 16
for (let i = 0; i < 16; i++) {
    // Start rendering at a given offset and render 128 samples
    synth.process(outL, outR, i * 128, 128);
}
Copy

Check out SpessaSynthProcessor.process for more information.

Examples

You can find all examples in the examples directory in this repository.

MIDI to WAV converter

Here is what this code does:

• Import necessary modules
• Read the files passed to the command line
• Parse the binary file buffers
• Initialize the synthesizer
• Initialize the sequencer
• Initialize the output buffers
• Render loop:
  • Process sequencer
  • Render BUFFER_SIZE samples into the output buffers
  • Log progress
• Convert to WAV and save

import * as fs from "node:fs/promises";
import {
    audioToWav,
    BasicMIDI,
    SoundBankLoader,
    SpessaLog,
    SpessaSynthProcessor,
    SpessaSynthSequencer
} from "../src";

// Process arguments
const args = process.argv.slice(2);
if (args.length !== 3) {
    console.info(
        "Usage: tsx index.ts <soundbank path> <midi path> <wav output path>"
    );
    process.exit();
}
// Read MIDI and sound bank
const sf = await fs.readFile(args[0]);
const mid = await fs.readFile(args[1]);
// Parse the MIDI and sound bank
const midi = BasicMIDI.fromArrayBuffer(mid.buffer);
const soundBank = SoundBankLoader.fromArrayBuffer(sf.buffer);

// Initialize the synthesizer
const sampleRate = 48_000;
const synth = new SpessaSynthProcessor(sampleRate, {
    eventsEnabled: false
});
synth.soundBankManager.addSoundBank(soundBank, "main");
await synth.ready;
// Enable verbose information during render
SpessaLog.setLogLevel(true, true, true);
// Enable uncapped voice count
synth.setSystemParameter("autoAllocateVoices", true);

// Initialize the sequencer
const seq = new SpessaSynthSequencer(synth);
seq.loadNewSongList([midi]);
seq.play();

// Prepare the output buffers
const sampleCount = Math.ceil(sampleRate * (midi.duration + 2));
const outLeft = new Float32Array(sampleCount);
const outRight = new Float32Array(sampleCount);
const start = performance.now();
let filledSamples = 0;
// Note: buffer size is recommended to be very small, as this is the interval between modulator updates and LFO updates
const BUFFER_SIZE = 128;
let i = 0;
const durationRounded = Math.floor(seq.midiData!.duration * 100) / 100;
while (filledSamples < sampleCount) {
    // Process sequencer
    seq.processTick();
    // Render
    const bufferSize = Math.min(BUFFER_SIZE, sampleCount - filledSamples);
    synth.process(outLeft, outRight, filledSamples, bufferSize);
    filledSamples += bufferSize;
    i++;
    // Log progress
    if (i % 100 === 0) {
        console.info(
            "Rendered",
            Math.floor(seq.currentTime * 100) / 100,
            "/",
            durationRounded
        );
    }
}
const rendered = Math.floor(performance.now() - start);
console.info(
    "Rendered in",
    rendered,
    `ms (${Math.floor(((midi.duration * 1000) / rendered) * 100) / 100}x)`
);
const wave = audioToWav([outLeft, outRight], sampleRate);
await fs.writeFile(args[2], new Uint8Array(wave));
console.info(`File written to ${args[2]}`);
Copy