Re: [Tinycc-devel] make tcc reentrant

[grischka]

uso ewin wrote:
> On Wed, Dec 11, 2019 at 3:44 PM Michael Matz <address@hidden> wrote:
> > Hi,
> >
> > On Wed, 11 Dec 2019, grischka wrote:
> >
> > > Couldn't help to try out some things, related to the matter.
> > > See
> > > https://repo.or.cz/tinycc.git/commitdiff/72729d8e360489416146d6d4fd6bc57c9c72c29b
> > Yeah, something like that was in my mind as well.
> >
> > > Anyway I think, it still does look like tinycc.
> > Indeed, nice, I like it!  But let me disagree with parts of one comment
> > you added:
> >
> > +    /* Here we enter the code section where we use the global variables for
> > +       parsing and code generation (tccpp.c, tccgen.c, <target>-gen.c).
> > +       Other threads need to wait until we're done.
> > +
> > +       Alternatively we could of course pass TCCState *s1 everwhere
> > +       except that it would look extremly ugly.
> > +
> > +       Alternatively we could use thread local storage for those global
> > +       variables, which may or may not have advantages */
> >
> > The last alternative really isn't one.  If you move some global variables
> > to be thread-local, you now have multiple potentially different copies.
> > Some TCCState state refers to the global (then thread-local) variables
> > (e.g. symbol table or string/token indices), so you'd have to associate
> > each TCCState with the specific instance of the thread-local to which it
> > wants to refer.  Now you either have bound each TCCState to a specific
> > thread, or you need to introduce locking again because you might refer to
> > data from different threads, which thread-locality was supposed to avoid.
> > The only reason why the current scheme works is because there's only a
> > single copy of the global data.  Having multiple copies (e.g. by
> > introducing thread-local) effectively means you have to move that data
> > into TCCState itself (at which point thread-local become moot again).
> >
> >
> > Ciao,
> > Michael.

> I'm pretty happy with the solution found so I wont complain but I'm not sure to
> understand the problem with Thread local storage.
> If I've understand correctly the problem happen only if you use a TCCState
> that has been allocated in another thread, is that right ?
> Is it really a problem ? (you can return an error if running from the
> wrong thread)
> Well in any case the semaphore seems a cleaner solution

I assume Michael wrote that under the impression that there is still
global data with lifetime beyond the tcc_compile function call, which
was the case for example for the tokens and token hash table.

I have to admit that I did remove that feature,  such that basically
there is no relation whatsoever between the global data used by
tcc_compile and the TCCState.

Under this circumstances I don't see a way for other threads to mess
with the data (while it is valid) if they are either blocked to enter
this function or are given a different instance of the data automatically.

Whether and when that has what impact on performance and whether we
should care I'm not quite sure yet.  I've tested compiling cc1 from
gcc 3.4.6 directly from all 172 files to the exe in one go. Was 5.8,
now is 6.0 secs, maybe.

--- grischka

> Matthias,