Re: [fluid-dev] Making MIDI player read from a buffer

[Matt Giuca]

Hi David, James, Pedro,

Thanks for a quick discussion.

David Henningson wrote:

> ...although this would only be in fluid_midi.c, right? It wouldn't affect 
> other files.

Correct.

> I'd personally prefer the client handling memory entirely. This has two 
> advantages:
>  * We won't have to do "free" in a context that might be real-time sensitive.
>  * The client does not have to use the same memory allocator as fluidsynth.

OK, I thought about this when I was typing my first email, but I
realised there would be a major drawback in that without a garbage
collector, the client would need to somehow know when it is safe to
free the memory. As far as I can tell, you can really only be sure the
fluid_player is finished with it if the entire playlist has ended, or
you have manually stopped or deleted the fluid_player. Therefore if
you were creating a playlist from some file-like objects, you would
need to load them all into memory, store pointers to each of them
somewhere, wait for the fluid_player to end, then free them all.

It seems in almost all cases I can think of, the client is going to
want to allocate a buffer specially for FluidSynth to read, populate
it from whatever file-like object it has in mind, and then pass it to
FluidSynth, never to read the buffer (in the client) again. There
would be a lot of overhead if all the clients had to do manual garbage
collection on the buffers.

As for the client having to use the same memory allocator, true, but
as I said most clients will simply quickly allocate a buffer which is
immediately passed to Fluid and forgotten about, so even if the client
is using another allocator, it could just use malloc at this one part.
For example, a C++ program would typically use new[] to construct
arrays, but it would use malloc to allocate this particular buffer to
pass to FluidSynth. (Though I agree it's not too nice for a library to
free a data structure allocated by the client.)

I don't know much about real-time programming. But I imagine that if
you are loading MIDI files into memory each time you hit a new
playlist entry, you have lost your real-time-ness anyway (at least, in
between songs), since you will be allocating a new internal data
structure, reading a file from disk, and freeing the previous one.
Does adding this free make matters any worse for real-time
applications?

A possible work-around for this is having fluid_player_add_mem take an
extra argument of type void(*)(void*) (a function pointer which
accepts a void* and returns void), which is called on the buffer when
the player is done with it. This lets you specify your own
de-allocation function (for example, C++ clients could pass a function
which calls delete[] on a new[]-allocated array), or pass a function
which does nothing, allowing clients to ignore the signal and perform
their own memory-management. But the majority of clients could just
pass 'free' as an argument, which means FluidSynth would simply free
the memory with no further work on the part of the client. Or maybe
that's going too far.

>
> In addition, here are things that might or might not be real-time sensitive. 
> This conversion sounds like time intensive.

OK, again I don't know much about real-time programming, so I might
have to get someone to check my code afterwards to let me know if I've
"broken" some real-time-ness of FluidSynth.

> Here's a compromise:
>
> struct loadable_file
> {
>        char* filename;
>        void* data;
>        size_t data_length;
> }
>
> (You don't need an enum because either one will be NULL.)
>
> But then, at file load time, if filename != NULL, load the file into the data 
> and data_length fields, then do midi parsing, followed by freeing the data 
> buffer again. Then we at least have only one midi file in memory at a time.

OK that does seem like a good solution. Note however that before when
you said "That is probably a good thing though," (referring to the
fact that you get errors upon calling fluid_player_add rather than at
load time), that is no longer the case -- with this design you would
get file access errors as each file comes up in the playlist.

> The other end of it would be to run through all of the midi conversion 
> functions inside fluid_player_add_mem. With all time intensive tasks done, we 
> might be able to switch from one midi file to another very quickly (without 
> risking underruns). That'd be nice, but I don't know about the memory 
> consumption.

You mean fully parse the MIDI file into track data in
fluid_player_add_mem. I see, well I don't think I'll tackle that just
yet, but I agree it could be an interesting trade-off (somewhat
increased memory consumption, but having no overhead -- disk or memory
-- in switching MIDI files).

Well, garbage collection issues notwithstanding, I think I'll start
implementing the "compromise" method you describe. I will share the
code once I have something,

Matt