Sometimes, it is necessary for the script to have direct access to the synthesizer's audio engine for various reasons.
This page shows how to use spessasynth_core's SpessaSynthProcessor and SpessaSynthSequencer directly,
rendering in the main thread with full access to the audio engine.
A simple audio loop that achieves this is as follows:
Float32Array buffers for the stereo output (and optionally the per-channel outputs for visualization).AudioBufferSourceNode.Below is an example that shows the current channel status, which is something that cannot be achieved with just the WorkletSynthesizer class.
<label for="sound_bank_input">Upload the sound bank.</label>
<input accept=".sf2, .sf3, .dls" id="sound_bank_input" type="file" />
<label for="midi_input">Select the MIDI file</label>
<input accept=".midi, .mid, .rmi, .smf" id="midi_input" type="file" />
<h2>Voice list</h2>
<div
id="voice_list"
style="display: flex; width: 100%; justify-content: space-evenly"
></div>
<!-- note the type="module" -->
<script src="main_thread_rendering.js" type="module"></script>
Nothing special here.
import {
BasicMIDI,
SoundBankLoader,
SpessaSynthProcessor,
SpessaSynthSequencer
} from "spessasynth_core";
// This demo shows how to render in the main thread in real time
// Use firefox for this, chromium poorly handles audio buffers being used like this
// For chromium, consider making a simple playback worklet processor instead
// Create a new audio context
const context = new AudioContext({
sampleRate: 44_100
});
// Wait for the user to upload the sound bank
document
.querySelector("#sound_bank_input")!
.addEventListener("change", async (event) => {
// If no file is selected, exit early
const files = (event.target as HTMLInputElement)?.files;
if (!files?.[0]) {
return;
}
// Resume the audio context so audio processing can begin
await context.resume();
// Read the uploaded file into an ArrayBuffer
const fontBuffer = await files[0].arrayBuffer();
// Create an instance of the synthesizer and load it with the sound bank
const synth = new SpessaSynthProcessor(44_100);
synth.soundBankManager.addSoundBank(
SoundBankLoader.fromArrayBuffer(fontBuffer),
"main"
);
// Initialize the sequencer for MIDI playback
const seq = new SpessaSynthSequencer(synth);
// THE MAIN AUDIO RENDERING LOOP IS HERE
setInterval(() => {
// Get the synthesizer’s internal current time
const synTime = synth.currentTime;
// If the synth time is significantly ahead of the context time, skip rendering
// (wait for the context to catch up)
if (synTime > context.currentTime + 0.1) {
return;
}
// Create empty stereo buffers for the output (effects are already mixed in)
const QUANTUM = 128;
const BUFFER_SIZE = 2048;
const outputL = new Float32Array(BUFFER_SIZE);
const outputR = new Float32Array(BUFFER_SIZE);
let rendered = 0;
while (rendered < BUFFER_SIZE) {
// Play back the MIDI file
seq.processTick();
// Render the next chunk of audio into the provided buffers
synth.process(outputL, outputR, rendered, QUANTUM);
rendered += QUANTUM;
}
// Create an AudioBuffer to hold the sample data
const outBuffer = new AudioBuffer({
numberOfChannels: 2,
length: BUFFER_SIZE,
sampleRate: 44_100
});
// Copy the left and right channel data into the audio buffer
outBuffer.copyToChannel(outputL, 0);
outBuffer.copyToChannel(outputR, 1);
// Create a source node from the buffer and connect it to the desired output
const source = new AudioBufferSourceNode(context, {
buffer: outBuffer
});
source.connect(context.destination);
// Schedule the buffer to play at the synth's current time
source.start(synTime);
}, 0);
// List all the voices currently playing
const list = document.querySelector("#voice_list")!;
// Create and store a <pre> element for each of the 16 MIDI channels
// Each one will be used to display information about active voices on a given channel
const voiceListElements: HTMLPreElement[] = [];
for (let index = 0; index < 16; index++) {
const element = document.createElement("pre");
voiceListElements.push(element);
list.append(element);
}
// Set up an interval to regularly update the voice display for each channel
setInterval(() => {
// Note: this code is working directly with the synth engine.
// Advanced users only.
const core = (
synth.midiChannels[0] as unknown as {
synthCore: {
voices: {
isActive: boolean;
channel: number;
midiNote: number;
}[];
};
}
).synthCore;
// Start building the display string with the channel number
const textData = voiceListElements.map(
(_, chanNumber) => `Channel ${chanNumber + 1}:\n`
);
for (const voice of core.voices) {
if (!voice.isActive) continue;
// Get the corresponding element for this channel
// Append a line for each currently active voice with its MIDI note
textData[voice.channel] += `note: ${voice.midiNote}\n`;
}
for (const [
index,
voiceListElement
] of voiceListElements.entries()) {
voiceListElement.textContent = textData[index];
}
}, 100);
// Set up the MIDI player
document
.querySelector("#midi_input")!
.addEventListener("change", async (event) => {
// Verify if the file is really there
const input = event.target as HTMLInputElement;
const file = input.files?.[0];
if (!file) {
return;
}
// Parse and play the file
const midi = BasicMIDI.fromArrayBuffer(
await file.arrayBuffer()
);
seq.loadNewSongList([midi]);
seq.play();
});
});
The audio loop presented in this script is very similar to the one shown above:
There is another loop that displays all the voices. It is independent of the audio loop.