Re: [bug #40639] GNU Make with profiling information

[Eddy Petrișor]

Pe 29.11.2013 12:30, "Tim Murphy" <address@hidden> a scris:
>
> When I did something similar (which I am not allowed to post) I made a
> new file for each submake and the output filename "base" was in an
> environment variable.  I realise that nobody ever wants to change the

Does it make sense to separate the profiling info into one file per process and then try to merge the information back?

The second proposal I sent prints the pid for the 'short' and 'long' formats, so if one wants to focus on a specific submake, one can grep for the pid in the whole profile listing. The first proposal had a flexible output format, so one could choose to print the pid info then filter that.

> way they implemented something but I thought I'd mention it.
>
> Also, having written a few external tools, I always appreciate it when
> the creator of some information source makes it easy for me to recover
> data and the relationships between each item.  I also like it when

Not sure what you are suggesting here. What relation would you want and think is not available?

> there's not really any way for my parser to potentially make some
> mistake just because a tool prints out something similar to what I'm

The first version allowed a custom format which allowed any literal string to be printed anywhere, alongside with the target name, start and end timestamps, pid duration and recursivity level.

The second proposal allows a custom prefix before the profiling info.

So in both cases a simple grep for a specific prefix which can be chosen by the user is possible. Do you think that would be enough for your hypothetical parser?

> looking for.  I also like it when the format is such that it can
> change a bit without necessarily forcing me to rewrite my parser.

Not sure what would you think could change in the output without informing the parser at all.

Currently the prefix and the field separators are customisable.

Maybe an example could help.

>
> Again, this is only an opinion - it carries no weight.
>
> Regards,
>
> Tim
>
> On 29 November 2013 02:43, Eddy Petrișor <address@hidden> wrote:
> >
> > Pe 27.11.2013 13:12, "Tim Murphy" <address@hidden> a scris:
> >
> >
> >>
> >> FWIW
> >>
> >> As for profiling output, this should probably go to a file (possibly
> >> with a .PID on the end) , not stdout .....unless..... you start to
> >> embrace the idea of structured output for everything that make
> >> produces.
> >
> > The profiling info is printed on stderr, not on stdout.
> > I also initially thought I should isolate the information in a different
> > file, but the problem is that for recursive make that file descriptor must
> > be passed on every fork, which is a very intrusive change to make and very
> > wasteful (passing an extra file descriptor, even when unnecessary).
> >
> > So I opted for the next best thing, prefixing the output with something a
> > grep could anchor to.
> >
> >>
> >> I have used XML before and it has advantages, not the least of which
> >> is that it is somewhat robust in the face of corruption and that does
> >> happen. I prefer JSON though because it's more readable and it's very
> >> easy to parse.
> >
> > As I said before, I think that's the job of an external tool.
> >
> >>
> >> On 27 November 2013 07:56, Eddy Petrișor <address@hidden> wrote:
> >> >
> >> > Pe 25.11.2013 11:09, "Reinier Post" <address@hidden> a scris:
> >> >
> >> >
> >> >> > > Can't this functionality be provided by a wrapper $SHELL?
> >> >> > > Sure, it's an extra exec(),
> >> >> > > but it will keep the make code base simpler.
> >> >> >
> >> >> > I'm not sure if I understand what you mean. Do you mean wrapping all
> >> >> > target invokes in a $SHELL?
> >> >>
> >> >> Yes; something line /usr/bin/time.
> >> >
> >> > Not on Windows.
> >> >
> >> >> > If that's the case, you don't get the actual/real-time measurement,
> >> >> > and
> >> >> > that might skew your profiling information.
> >> >>
> >> >> True.  Hnow how much of a difference does it make?
> >> >
> >> > Depends on how badly the build system is written. And that's what the
> >> > target
> >> > of the profiling is, finding out how bad is it. I feel that penalising
> >> > the
> >> > most exactly the build systems that need the least the slow down it's
> >> > bad
> >> > design.
> >> >
> >> >> > I have some build systems that can take easily over 12 hours to do a
> >> >> > full build (several thousands of elfs in recursive
> >> >> > make calls - I know that's bad, I am working on it), so any extra
> >> >> > time
> >> >> > waste just for the measurement can mean
> >> >> > significant delays in getting the results.
> >> >>
> >> >> If it's 15 minutes, they are not the thing you want to optimize first.
> >> >> The build is probably running overnight anyway.
> >> >
> >> > If it's a live/production system, then any unknown slow down time might
> >> > not
> >> > be acceptable.
> >> >
> >> >
> >> > _______________________________________________
> >> > Bug-make mailing list
> >> > address@hidden
> >> > https://lists.gnu.org/mailman/listinfo/bug-make
> >> >
> >>
> >>
> >>
> >> --
> >> You could help some brave and decent people to have access to
> >> uncensored news by making a donation at:
> >>
> >> http://www.thezimbabwean.co.uk/friends/
>
>
>
> --
> You could help some brave and decent people to have access to
> uncensored news by making a donation at:
>
> http://www.thezimbabwean.co.uk/friends/