Fwd: Re: the files listed with du -a do not match ls -R, possible bug?

[Aaron Wegner]

----------  Forwarded Message  ----------

Subject: Re: the files listed with du -a do not match ls -R, possible bug?
Date: Mon, 19 Aug 2002 09:33:26 -0500
From: Aaron Wegner <address@hidden>
To: address@hidden (Bob Proulx)

On Saturday 17 August 2002 12:51 pm, you wrote:
> Aaron Wegner <address@hidden> [2002-08-15 13:59:06 -0500]:
> > I think I found what may be a bug in the du program.  So far I have
> > only been able to observe this anomaly in my build directory for
> > glibc.
>
> I as well was unable to recreate your problem in any of the
> directories that I had lying around.  BTW that was a good job of
> presenting a small test case of commands that would recreate the
> problem.  However the data set is not readily available so more help
> is needed.

Yes, this is an unusual problem, and one that I haven't experimented with
 very much.  Files with a .op extension are object files with extra code
 inserted to write profile information suitable for the analysis program
 gprof.  The makefiles for glibc automatically build these object files with
 normal builds consisting of "configure" and "make".  Actually, three object
 files for every .c source file are created: one with a .o extension for use
 in static archives such as libc.a, one with a .os extension for use in
 shared libraries such as libc.so, and one with a .op extension for gprof. 
 There are many options passed to gcc during the compilation, and most are
 common to all three builds, but the differences are listed below.

-o file.o
-pg -DPROF -o file.op
-fPIC -DPIC -DSHARED -o file.os

The thing is, these .op files are not grouped into a convenient archive, such
as libc.a, during the build process.  That's what I was trying to do, and how
I discovered the bug.  I was trying to pull together all the .op files into
one huge archive that I could use to statically link into my program, so that
I could see what C library functions were being called by my program, and how
much time my program was spending in the subroutines, using the analysis
program gprof.  After getting some unresolved symbols and linker errors, I
discovered that du -a was not listing all the files (I was using du -a to
generate the list of files.)  The three files that it omits, on my computer,
and on my coworker's, are:

-rw-r--r--    2 root     root        15544 Aug  8 16:06 gmon/mcount.op
-rw-r--r--    2 root     root        36184 Aug  8 16:06 gmon/sprofil.op
-rw-r--r--    2 root     root        26040 Aug  8 16:06 gmon/profil.op

Ironically, the files are located in the subdirectory whose object files are
responsible for updating the data structures that gprof uses.  I guess that
the main problem with observing this bug is that you have to have a build
directory for glibc, and this is probably one of the longest builds that I am
aware of, something like 2 hours.  I tried the same sequence of commands on
files with .c extensions, but I didn't find any discrepancies.  So, I don't
know why it isn't finding these object files, and I haven't thoroughly
experimented to find another instance of the bug.

Thanks,

AW

> > The bug is uncovered with the following commands, issued
> > from the base of the [glibc] build directory:
> > --------------------------------------
> > # ls -lR | grep '\.op$' | awk '{printf("%s\n", $NF)}' | sort > ls_c_files
> >
> > # du -a . | grep '\.op$' | awk 'BEGIN {FS="[\/]"} {printf("%s\n", $NF)}'
> >
> > | sort > du_c_files
> >
> > # sdiff -s du_c_files ls_c_files
> >
> >                                                               > mcount.op
> >                                                               > profil.op
> >                                                               > sprofil.op
> >
> > ---------------------------------------
>
> What types of files are those *.op files?
>
>   ls -ld *.op
>
> I am guessing they are not normal files.  Probably dangling symlinks
> or sockets or some such.  But it would be a clue as to why those files
> were not seen by 'du -s'.
>
> Thanks
> Bob

-------------------------------------------------------