[fluid-dev] Making MIDI player read from a buffer

[Matt Giuca]

There was a discussion in the thread "FluidSynth backend for
SDL_mixer" about the difficulty with the current implementation of the
FluidSynth API's MIDI file player, in that it will only load a MIDI
file via a filename (fluid_player_add --
http://fluidsynth.sourceforge.net/api/midi_8h.html#a1ac5b59e4ab9d0f48e917dda0a9a4403
-- seems to be the only way to load anything into the MIDI player).
This means if you need to load a MIDI file from any source that is not
a regular disk file (e.g., from an already-open FILE* such as stdin, a
network socket, an SDL RWop, or another high-level "file-like object"
which is not an ordinary file on disk), you are presently out of luck.

This apparently was so bad that James Le Cuirot, working on SDL_mixer
backend for FluidSynth, wrote a whole MIDI parser ...

In that thread, I volunteered to help fix FluidSynth around this
(thanks to James for reminding me this morning). It looks like quite a
job since the "loading of files via a filename" aspect goes quite deep
into the internals of FluidSynth. I thought I'd start this thread to
a) see if there is interest in this change, and b) have a discussion
about whether I'm going about it right. I think it would be quite
useful to fix. What I proposed was not the most general solution to
this problem (the "ideal" solution would be to allow any "file-like
object" to be plugged into FluidSynth and have it request bytes from
that), but a good compromise: Allow FluidSynth to read MIDI file data
from a void* memory buffer. Since MIDI files are small, any other
"file-like object" could simply load all of the bytes into memory and
then pass the void* buffer in. Like I said, it's not ideal, but at
least it allows these solutions to function.

I can hopefully do this without breaking the existing FluidSynth API,
but it would require a lot of changes to the internals. Here we go.

1. INTERFACE CHANGES

There is simply one new function, declared in
include/fluidsynth/midi.h and defined in src/midi/fluid_midi.c:

/**
 * Add a MIDI file to a player queue, from a buffer in memory.
 * @param player MIDI player instance
 * @param buffer Pointer to memory containing the bytes of a complete MIDI
 *   file. The pointer is "stolen" and will be freed by FluidSynth once it is
 *   no longer needed.
 * @param len Length of the buffer, in bytes.
 * @return #FLUID_OK
 */
FLUIDSYNTH_API int fluid_player_add_mem(fluid_player_t* player, void
*buffer, size_t len);

The idea is that it is possible to add filenames (using
fluid_player_add) and memory buffers (using fluid_player_add_mem)
interleaved in the queue, so each queue item can either be a filename
entry or a memory buffer entry.

2. INTERNAL CHANGES

Internally, this will be difficult, as the following data structures
are currently hard-wired to deal with filenames/FILE*s (from
src/midi/fluid_midi.h):

/*
 * fluid_player
 */
struct _fluid_player_t {
  ...
  fluid_list_t* playlist; /* List of file names */
  fluid_list_t* currentfile; /* points to an item in files, or NULL if
not playing */
  ...
}

/*
 * fluid_midi_file
 */
typedef struct {
  fluid_file fp;
  ...
} fluid_midi_file;

(where fluid_file is a typedef for FILE*).

First, fluid_player_add adds the filename to fluid_player::playlist
(without loading from disk). This stores the filename as a char* in
playlist->data. Then, when it gets up to that particular playlist
item, it calls fluid_player_load (fluid_midi.c) with that filename,
which opens the file into a FILE*. This then temporarily creates a
fluid_midi_file object, storing the FILE*, and then calls a bunch of
other files to read the bytes of the MIDI file from disk and assemble
them into an in-memory data structure inside the fluid_player_t. It
then deletes the fluid_midi_file object, and from that point onwards,
everything is in memory (our job is done).

All of the above architecture would have to be modified to handle
reading from a memory buffer. (Note: Short of using mmap and doing
file descriptor tricks, which I'm sure would be Linux-specific or
something, there's not really any way to make use of the FILE*
infrastructure.) Possibly the easiest way to handle this would be to
*completely replace* the existing FILE* stuff with a void* instead,
and replace such functions as fluid_midi_file_getc with new
implementations which read bytes from the buffer. That would be
relatively straightforward. Then fluid_player_add_mem would work
naturally with the new infrastructure. The old fluid_player_add
function would then wrap around that -- it would load the file from
disk into a buffer in memory, then pass that buffer to
fluid_player_add_mem. However, I expect this will not be what people
want to do, because it fundamentally changes the way fluid_player_add
works -- it would load the entire file into memory up-front, rather
than the current approach which reads in a few bytes at a time, never
loading the whole file into memory. Maybe this is an acceptable
trade-off though (since come on, MIDI files are easily going to fit
into memory these days, even on embedded systems, and load very
quickly also). One important caveat is that it will mean file access
errors come up during fluid_player_add, rather than when the player
reaches the particular playlist item.

The other approach I can think of involves keeping the old
architecture AND having the new one. It's quite complicated.
Basically, fluid_player changes such that the playlist no longer
contains char* elements. It contains a new type:

struct loadable_file
{
    enum { FILENAME, BUFFER } tag;
    union { char*, void* } data;
    size_t data_length;
}

which can contain either the tag FILENAME and a char* with the
filename (and no length), or the tag BUFFER and a void* pointing to
the buffer, as well as the size_t for the data length.

Upon loading the file, if the tag is FILENAME, it will open the file
into a file pointer. If it is BUFFER it will not change anything. Then
it will copy that into another new type stored in the fluid_midi_file
where the fp currently is:

struct loaded_file
{
    enum { FILE, BUFFER } tag;
    union { fp, void* } data;
    size_t data_length;
}

This is the same idea as loadable_file, with one difference: instead
of a filename it has a fp (FILE*).

Now all of the (handily abstracted) file-like operations such as
fluid_midi_file_getc will switch on the tag, and if it is a FILE, call
getc(), and if it is a BUFFER, read bytes from memory, for example.
Now we have a system which works for files (reading in one byte at a
time) OR buffers.

That second approach is quite a lot of work.

So my questions for any interested observers (especially FluidSynth developers):

1. Is this feature worth implementing at all?
2. Is it acceptable to use the first solution (changing the way
fluid_player_add performs by loading the entire file into memory ahead
of time, rather than what it presently does which is to open a FILE*
at the time the song itself is played, reading the bytes one at a time
and never all into memory)? If so, that would be nice and easy.
3. If not, is it worth going to the hassle of adding all that
union-data-type stuff to support both forms?
4. Is there a better way than the madness I propose?

Cheers
Matt