RtMidi
RtMidi enables ffgac, ffedit, and fflive to use
MIDI controllers as input devices.
The "rtmidi" module is built-in to the quickjs engine.
It is a simple wrapper around RtMidi functionality.
It can be imported with:
import * as rtmidi from "rtmidi";
The "rtmidi" module provides the
In() constructor to create RtMidiIn objects, and the
Out() constructor to create RtMidiOut objects.
This page is mostly based on RtMidi documentation itself.
NOTE: Currently, you need to explicitly poll for the events.
I plan on moving this to using
async functions
in the future.
NOTE2: There is no support for RtMidi’s setCallback(), in case
you are reading RtMidi’s documentation and wondering why it’s not
here on this page.
RtMidiIn Constructor
The new rtmidi.In() constructor is used to create a new RtMidiIn object.
Syntax
new rtmidi.In()
Return value
The new RtMidiIn object.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
RtMidiIn.prototype.getVersion()
A static function to determine the current RtMidi version.
Syntax
getVersion()
Return value
A string with the current RtMidi version.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
print(midiin.getVersion());
RtMidiIn.prototype.getCompiledApi()
A static function to determine the available compiled MIDI APIs.
Syntax
getCompiledApi()
Return value
An Array with the IDs of the compiled MIDI APIs.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
const apis = midiin.getCompiledApi();
print(apis);
RtMidiIn.prototype.getApiName()
Return the name of a specified compiled MIDI API.
Syntax
getApiName(api)
Parameters
api is the specified MIDI API.
Return value
A string with the name of the specified MIDI API.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
const apis = midiin.getCompiledApi();
const first_api_name = midiin.getApiName(apis[0]);
print(first_api_name);
RtMidiIn.prototype.getApiDisplayName()
Return the display name of a specified compiled MIDI API.
Syntax
getApiDisplayName(api)
Parameters
api is the specified MIDI API.
Return value
A string with the display name of the specified MIDI API.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
const apis = midiin.getCompiledApi();
const first_api_display_name = midiin.getApiDisplayName(apis[0]);
print(first_api_display_name);
RtMidiIn.prototype.getCompiledApiByName()
Return the compiled MIDI API having the given name.
Syntax
getCompiledApiByName(name)
Parameters
name is the specified MIDI API name.
Return value
A number with the specified MIDI API.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
const apis = midiin.getCompiledApi();
const first_api_name = midiin.getApiName(apis[0]);
const first_api = midiin.getCompiledApiByName(first_api_name);
// first_api should be the same value as apis[0]
RtMidiIn.prototype.setClientName()
This function sets the client name (I don’t really know what this means).
Syntax
setClientName(name)
Parameters
name is the name for the client.
Return value
true
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
midiin.setClientName("client");
RtMidiIn.prototype.setPortName()
This function sets the port name (I don’t really know what this means).
Syntax
setPortName(name)
Parameters
name is the name for the port.
Return value
true
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
midiin.setPortName("port");
RtMidiIn.prototype.getCurrentApi()
Returns the MIDI API specifier for the current instance of RtMidiIn.
Syntax
getCurrentApi()
Return value
A number with the current MIDI API specifier for the current instance of RtMidiIn.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
const current_api = midiin.getCurrentApi();
print(current_api);
RtMidiIn.prototype.getPortCount()
Return the number of available MIDI input ports.
Syntax
getPortCount()
Return value
A number with the number of available MIDI input ports.
Throws an exception on error.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
const port_count = midiin.getPortCount();
print(`port_count ${port_count}`);
RtMidiIn.prototype.getPortName()
Return a string identifier for the specified MIDI input port number. Throws an exception on error.
Syntax
getPortName(portNumber)
Parameters
portNumber is an optional port number greater than 0 can be specified.
Return value
A string identifier for the specified MIDI input port number.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
const port_count = midiin.getPortCount();
print(`port_count ${port_count}`);
for ( let i = 0; i < port_count; i++ )
{
const port_name = midiin.getPortName(i);
print(`[${i}] ${port_name}`);
}
RtMidiIn.prototype.openPort()
Open a MIDI input connection given by enumeration number.
Syntax
openPort(portNumber, portName)
Parameters
portNumberis an optional port number greater than 0 can be specified. Otherwise, the default or first port found is opened.portNameis an optional name for the application port that is used to connect to portId can be specified.
Return value
true on success.
Throws an exception on error.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
midiin.openPort();
RtMidiIn.prototype.openVirtualPort()
Create a virtual input port, with optional name, to allow software connections.
Syntax
openVirtualPort(portName)
Parameters
portName is an optional name for the application port that is used to connect to portId can be specified.
Return value
true on success.
Throws an exception on error.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
midiin.openVirtualPort();
RtMidiIn.prototype.closePort()
Close an open MIDI connection (if one exists).
Syntax
closePort()
Return value
true on success.
Throws an exception on error.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
midiin.openPort();
midiin.closePort();
RtMidiIn.prototype.isPortOpen()
Returns true if a port is open and false if not.
Syntax
isPortOpen()
Return value
A boolean with true if a port is open and false if not.
Throws an exception on error.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
midiin.openPort();
const is_port_open = midiin.isPortOpen();
print(`is_port_open ${is_port_open}`);
midiin.closePort();
RtMidiIn.prototype.ignoreTypes()
Specify whether certain MIDI message types should be queued or ignored during input.
Syntax
ignoreTypes(midiSysex, midiTime, midiSense)
Parameters
midiSysexmidiTimemidiSense
Return value
true on success.
Throws an exception on error.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
midiin.openPort();
midiin.ignoreTypes(false, false, false);
RtMidiIn.prototype.getMessage()
You should call this function repeatedly and process the resulting
message until the function returns an empty Array (msg.length === 0).
NOTE:
The MIDI message format varies from one device to another, and is
outside the scope of this documentation.
You can find more information on google, or pages
like this one.
Syntax
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
midiin.openPort();
midiin.ignoreTypes(false, false, false);
while ( true )
{
const msg = midiin.getMessage();
if ( msg.length === 0 )
break;
print(JSON.stringify(msg));
}
midiin.closePort();
Return value
An Array with the bytes from one message received from the MIDI device.
Throws an exception on error.
Examples
import * as rtmidi from "rtmidi";
const midiin = new rtmidi.In();
RtMidiOut Constructor
The new rtmidi.Out() constructor is used to create a new RtMidiOut object.
NOTE:
RtMidiOut is used to send messages to MIDI devices.
If you want to receive and send messages from a device, you will have
to open both an RtMidiIn and an RtMidiOut.
Syntax
new rtmidi.Out()
Return value
The new RtMidiOut object.
Examples
import * as rtmidi from "rtmidi";
const rtmidiout = new rtmidi.Out();
RtMidiOut.prototype.getCurrentApi()
Returns the MIDI API specifier for the current instance of RtMidiOut.
Syntax
getCurrentApi()
Return value
A number with the current MIDI API specifier for the current instance of RtMidiOut.
Examples
import * as rtmidi from "rtmidi";
const midiout = new rtmidi.Out();
const current_api = midiout.getCurrentApi();
print(current_api);
RtMidiOut.prototype.getPortCount()
Return the number of available MIDI output ports.
Syntax
getPortCount()
Return value
A number with the number of available MIDI output ports.
Throws an exception on error.
Examples
import * as rtmidi from "rtmidi";
const midiout = new rtmidi.Out();
const port_count = midiout.getPortCount();
print(`port_count ${port_count}`);
RtMidiOut.prototype.getPortName()
Return a string identifier for the specified MIDI output port number. Throws an exception on error.
Syntax
getPortName(portNumber)
Parameters
portNumber is an optional port number greater than 0 can be specified.
Return value
A string identifier for the specified MIDI output port number.
Examples
import * as rtmidi from "rtmidi";
const midiout = new rtmidi.Out();
const port_count = midiout.getPortCount();
print(`port_count ${port_count}`);
for ( let i = 0; i < port_count; i++ )
{
const port_name = midiout.getPortName(i);
print(`[${i}] ${port_name}`);
}
RtMidiOut.prototype.openPort()
Open a MIDI output connection given by enumeration number.
Syntax
openPort(portNumber, portName)
Parameters
portNumberis an optional port number greater than 0 can be specified. Otherwise, the default or first port found is opened.portNameis an optional name for the application port that is used to connect to portId can be specified.
Return value
true on success.
Throws an exception on error.
Examples
import * as rtmidi from "rtmidi";
const midiout = new rtmidi.Out();
midiout.openPort();
RtMidiOut.prototype.openVirtualPort()
Create a virtual output port, with optional name, to allow software connections.
Syntax
openVirtualPort(portName)
Parameters
portName is an optional name for the application port that is used to connect to portId can be specified.
Return value
true on success.
Throws an exception on error.
Examples
import * as rtmidi from "rtmidi";
const midiout = new rtmidi.Out();
midiout.openVirtualPort();
RtMidiOut.prototype.closePort()
Close an open MIDI connection (if one exists).
Syntax
closePort()
Return value
true on success.
Throws an exception on error.
Examples
import * as rtmidi from "rtmidi";
const midiout = new rtmidi.Out();
midiout.openPort();
midiout.closePort();
RtMidiOut.prototype.isPortOpen()
Returns true if a port is open and false if not.
Syntax
isPortOpen()
Return value
A boolean with true if a port is open and false if not.
Throws an exception on error.
Examples
import * as rtmidi from "rtmidi";
const midiout = new rtmidi.Out();
midiout.openPort();
const is_port_open = midiout.isPortOpen();
print(`is_port_open ${is_port_open}`);
midiout.closePort();
RtMidiOut.prototype.sendMessage()
Immediately send a single message out an open MIDI output port.
Syntax
sendMessage()
Return value
true on success.
Throws an exception on error.
Examples
import * as rtmidi from "rtmidi";
const midiout = new rtmidi.Out();
midiout.openPort();
midiout.sendMessage([ 0xF0, 0x00, 0xF7 ]);
midiout.closePort();
Full Example
This example will open the MIDI device specified by the script
parameter to pixelsort the image.
The pixelsorting threshold will be set with the MIDI controller using
the first 4 faders in pairs. The low threshold is fader[1] - fader[0]
and the high threshold is fader[3] - fader[2].
NOTE: Most MIDI controllers only send messages when a control
is changed. Therefore you will probably get no visible change
in the script below before you start playing around with the
faders.
import * as rtmidi from "rtmidi";
let midiin = null;
let midi_threshold_low = [ 0, 0 ];
let midi_threshold_high = [ 0, 0 ];
// options
const opt_pix_fmt = "gbrp"; // supported pixel formats are "gbrp" and "yuv444p"
const opt_colorspace = "hsv"; // yuv444p: "yuv";
// gbrp: "rgb", "hsv", "hsl"
const opt_range_y = [ 0, 1 ]; // [ [0 .. 1], [0 .. 1] ]
const opt_range_x = [ 0, 1 ]; // [ [0 .. 1], [0 .. 1] ]
const opt_threshold = [ 0.25, 0.75 ]; // [ [0 .. 1], [0 .. 1] ]
const opt_order = "vertical"; // "vertical" or "horizontal"
const opt_reverse = false; // true or false
const opt_trigger_by = 2; // 0, 1, or 2
const opt_sort_by = 2; // 0, 1, or 2
/*********************************************************************/
/* scales value from 'from' range to 'to' range */
function scaleValue(value, from_min, from_max, to_min, to_max)
{
return (value - from_min) * (to_max - to_min) / (from_max - from_min) + to_min;
}
/*********************************************************************/
export function setup(args)
{
// select pixel format
args.pix_fmt = opt_pix_fmt;
// initialize RtMidi and list ports
midiin = new rtmidi.In();
const portCount = midiin.getPortCount();
console.log(`rtmidi port count: ${portCount}`);
for ( let i = 0; i < portCount; i++ )
{
const name = midiin.getPortName(i);
console.log(`[${i}] ${name}`);
}
// parse params (a number is expected)
if ( !("params" in args) )
throw new Error("A parameter is expected for the MIDI port number in the command line (use -sp <port number>).");
const port = args.params;
const ok = typeof port === 'number'
&& Number.isInteger(port)
&& port >= 0;
if ( !ok )
throw("MIDI port number expected as script parameter");
// open the selected MIDI controller
if ( midiin.openPort(port) === null )
throw(`Error opening MIDI controller at port ${port}`);
console.log(`MIDI controller at port ${port} opened`);
midiin.ignoreTypes(false, false, false);
}
/*********************************************************************/
function parse_rtmidi_events()
{
while ( true )
{
const msg = midiin.getMessage();
const msglen = msg.length;
if ( msglen == 0 )
break;
// Uncomment the following line to debug the message structure.
// console.log(JSON.stringify(msg));
if ( msglen == 3 )
{
if ( msg[0] == 176 )
{
switch ( msg[1] )
{
/* faders */
case 0: midi_threshold_low [0] = msg[2]; break;
case 1: midi_threshold_low [1] = msg[2]; break;
case 2: midi_threshold_high[0] = msg[2]; break;
case 3: midi_threshold_high[1] = msg[2]; break;
}
}
}
}
}
/*********************************************************************/
export function filter(args)
{
// parse all RtMidi events
parse_rtmidi_events();
// input data
const data = args["data"];
const height = data[0].height;
const width = data[0].width;
// range
const y_begin = Math.lround(scaleValue(opt_range_y[0], 0, 1, 0, height));
const y_end = Math.lround(scaleValue(opt_range_y[1], 0, 1, 0, height));
const x_begin = Math.lround(scaleValue(opt_range_x[0], 0, 1, 0, width));
const x_end = Math.lround(scaleValue(opt_range_x[1], 0, 1, 0, width));
const range_y = [ y_begin, y_end ];
const range_x = [ x_begin, x_end ];
// colorspace: yuv444p: "yuv";
// gbrp: "rgb", "hsv", "hsl"
const colorspace = opt_colorspace;
// trigger_by: yuv444p: 'y', 'u', 'v';
// gbrp: 'r', 'g', 'b',
// 'h', 's', 'v',
// 'h', 's', 'l'
const trigger_by = opt_colorspace[opt_trigger_by];
// sort_by: yuv444p: 'y', 'u', 'v';
// gbrp: 'r', 'g', 'b',
// 'h', 's', 'v',
// 'h', 's', 'l'
const sort_by = opt_colorspace[opt_sort_by];
// threshold
const y_low = midi_threshold_low [1] - midi_threshold_low [0];
const y_high = midi_threshold_high[1] - midi_threshold_high[0];
const threshold_low = scaleValue(y_low, -127, 127, opt_threshold[0], opt_threshold[1]);
const threshold_high = scaleValue(y_high, -127, 127, opt_threshold[0], opt_threshold[1]);
const threshold = [ threshold_low, threshold_high ];
// options
const options = {
colorspace: colorspace,
trigger_by: trigger_by,
sort_by: sort_by,
order: opt_order,
mode: "threshold",
reverse_sort: opt_reverse,
threshold: threshold, // can be high low or low high
clength: 0,
};
// call the internal pixelsort function
// console.log(orig_frame_num, div_frame_num, JSON.stringify(options));
ffgac.pixelsort(data, range_y, range_x, options);
}
Run it with (replace the <port_number> with an actual port number, from the list printed by the script when no parameters are specified):
$ fflive -i input.avi -vf script="script.js:<port number>"
NOTE2: You can also use RtMidi with all scripts available in FFglitch.
This means normal bitstream transplication scripts, vf_script,
pict_type_script, mb_type_script, and so on…