Selim – a real-time musical score follower toolkit
You can provide Selim with
- a MIDI file (or text input with millisecond timestamps and integers for MIDI note-on numbers), and
- a human performance of the same music (on a real-time MIDI device, or as text input via a Unix socket, TCP socket, websocket or standard input)
The software will then do its best to figure out where the player is going in the given MIDI file.
It will be able to output in real time, based on user choices, e.g.:
- a time index in the MIDI file
- events from the original MIDI file, synchronized to the performance
- events from a second MIDI file, synchronized to the performance
- MIDI events from the human performer as pass-through
- ...whatever exciting we'll come up with
The software should be usable as
- a Rust library
- a Python extension
- a command line utility communicating via MIDI devices, Unix sockets, websockets, TCP sockets or standard I/O
Examples
$ cargo run --bin selim-mid-info piece.mid
Compiling selim v0.1.0 (/home/kaiant/prg/selim)
Finished dev [unoptimized + debuginfo] target(s) in 0.33s
Running `target/debug/selim-mid-info 'piece.mid'
midi file has 1 tracks!
first track has 16430 events!
first track has 446 'note on' events on channel 1!
Status and roadmap
- choose MIDI parser library
- ensure we can get data from MIDI files using the chosen MIDI parser (midly)
- function to turn a MIDI file into an in-memory reference score (ms+pitch)
- use only the first track which contains meaningful MIDI data
- use only the first channel which contains meaningful MIDI data
- use only "note on" events
- ignore velocity
- convert time offsets to milliseconds (disregarding tempo for now)
- tool to output reference score on stdout
- first naïve stateless score follower algorithm
selim-0.1.0- inputs:
- complete reference score (ms+pitch)
- reference time index at last previous input note (ms)
- reference time warp factor at last previous input note
- previous input notes (ms+pitch)
- new input notes (ms+pitch)
- outputs:
- reference time index at last new input note (ms)
- reference time warp factor at last new input note
- list of ignored new input notes (ms+pitch)
- support only monophony (order of events matters)
- ignore unexpected (wrong/extra) notes
- keep waiting for next correct note
- inputs:
- unit tests for
selim-0.1.0 - choose MIDI input library
- real-time tool to convert MIDI input into ms+pitch events on stdout
- real-time tool to test out
selim-0.1.x- inputs:
- reference MIDI file
- real-time ms+pitch events on stdin
- outputs on stdout:
- reference time index at last new input note (ms)
- reference time warp factor at last new input note
- ignored input notes
- inputs:
- wrong/missed/extra note tolerant score follower algorithm
selim-0.1.1- match new input notes with future reference notes within a time window
- jump directly to first matching note
- time warp factor adjustment limit in
selim-0.1.2 - refine MIDI file interpretation (ms+pitch+vel+dur)
- take tempo changes into account when converting to milliseconds
- include velocity
- convert "note off" events to durations
4.7k Jan 8, 2023
1.4k Dec 29, 2022
62 Oct 14, 2022
42 Nov 5, 2022
44 Oct 9, 2022
89 Dec 23, 2022
5k Jan 4, 2023
1.4k Dec 29, 2022
27 Nov 10, 2022
40 Nov 23, 2022
16 Sep 8, 2022
10 Oct 11, 2022
1 Nov 26, 2021
6 Mar 2, 2022
3.8k Jan 3, 2023
42 Nov 5, 2022
8 Nov 23, 2022
7 Apr 12, 2022
4 Sep 23, 2022
35 Oct 5, 2022
6 Dec 26, 2023
6 Jan 26, 2023
2 Jul 13, 2023
2 Apr 5, 2022
7.6k Dec 31, 2022