Re: Student for GSoC 2008 - procfs

[olafBuddenhagen]

Hi,

On Mon, Mar 24, 2008 at 09:11:20AM +0530, Madhusudan C.S wrote:

> > /proc/<pid>/mem is problematic. Do we really need it for procps etc?
[...]
>      If this was not the problem you were trying to mention please
>      tell me what it was.

Sorry -- I had only a stupid webmailer available, a slow internet
connection, and was in a hurry when writing this. Let me explain in
detail now.

I didn't mean to say that /proc/*/mem is problematic per se. I rather
meant that it's problematic to implement a Linux-compatible version in
the Hurd procfs translator.

The issue is that it's not possible to obtain all of the information
listed by the Linux variant. For one, in Linux, the file corresponding
the each mmap region is listed. In the Hurd, we have no central entity
keeping track of mmapped files. The kernel only knows about memory
objects. For each object, there is some process (called pager) backing
it -- usually some filesystem server. Mapping the memory object to a
file is entirely the individual pager's responsibility -- to obtain the
file name associated with a memory object, we would need a new interface
that allows asking the pager about it, or the actual process that
requested the mapping.

But that's not all. Linux procfs not only gives the path of the mapped
file, but also the inode and the major/minor device number of the
partition it is on. However, the Hurd doesn't have device numbers.
Rather, devices are described by the translators handling them. In case
of kernel-implemented devices, there is the Mach device name as
identifier, which is quite like major/minor device numbers, except that
it's an alphanumeric string rather than a pair of integers -- we can't
represent that in a compatible manner.

Filesystems can also live on devices not implemented by the kernel, and
thus having no device name at all. In some cases, the filesystem
location of the device translator could be used instead; but that's
again incompatible, and doesn't always exist either. Similary, some
filesystems on the Hurd are purely virtual (stuff like ftpfs for
example), and thus have no inodes.

The only way to unambiguously name any open file in the Hurd is to
provide the actual port. While it would be technically possible to do
that in a /proc filesystem I think, it would work totally different than
the /proc/*/mem on Linux...

The bottom line is that it's not possible to create a fully compatible
/proc/*/mem. But do we really need to? AFAIK this interface is used
mostly by debugging tools -- but debugging is very system-specific
anyways, and the tools need to be adopted to use whatever native
debugging interface the respective system provides; trying to be
compatible seems useless here.

Thus it's very important to find out whether /proc/*/mem is really
needed for ps, top and related tools; whether we need to implement it at
all, and if so, which parts...

> /proc/<pid>/cpu  -  Contains CPU state information of the process
> <pid> and exactly contains Current and last cpu in which it was
> executed.

I wonder what this is used for?

> /proc/<pid>/cwd -   Its a symlink which points to the current working
> directory of the process
> correcting the problems in /proc/<pid>/environ
>                         -   This contains values of Environment variables
> for that process. But the
>                              procfs doc in Hurd says it always fails. And
> the author says he doesn't
>                              know why. Have to work on it and fix it.

Would be nice to have these; but how important are they? Are they
necessary for ps and friends?

> completing /proc/<pid>/stat
>                         -    This gives the process status in a form not
> legible to humans. And the
>                               need of this is just this. Let me try to give
> an example thats given in the
>                               Nirendra's blog I had linked earlier.
>                               The 8th attribute in Linux's /proc/<pid>/stat
> is a per process flag which
>                               gives personal information of process. By
> doing a logical AND of per
>                               process flag and with the following values we
> can obtain the information
>                               thats given next to the values:
>                               0x00000002 Process being created
>                               0x00000004 Exiting
>                               0x00000008 Dead
>                               0x00000040 Process using super user privilage
>                               0x00000200 Process dumping core
>                               0x00000400 Process received some signal
>                               0x00000800 Process allocating memory
>                               0x00001000 Killed for out-of-memory

Sounds like something that ps and/or top might use...

> /proc/cmdline   -   Contains the command line arguments passed to Kernel
> /proc/swaps     -    Contains swap space utilization function.
> 
>      Also from additional searching and a bit of researching I have found
> that it would be nice if I do include these features in the list.
> 
> /proc/stat        -  Overall statistics about the system, such as the number
> of page faults
>                          since the system was booted.
> /proc/version   -   In the current implementation only gives the result of
> uname. But it would
>                          be nice if we provide other details provided by
> Linux versions such as gcc
>                          version used to compile the kernel and the uptime.
> /proc/uptime   -   It already seems to be implemented by looking at the
> code. If there are any
>                          problems and additional requirements I would
> consider it again

None of these are really related to providing information about
individual processes; I wouldn't consider them very important. We can
obtain some of these information rather easily -- but we shouldn't spend
too much effort on such things, unless ps or top or related tools
actually need them...

> It would also be nice if we can provide
> /proc/sys/* features like
> /proc/sys/kernel/* - which reflects general kernel behaviors.
> /proc/sys/net/*  - which contains all the network related information on the
> system.

Same here.

> One more important feature that would be desirable is /proc/<pid>/fd -
> which contain the symlinks to all the files the process has opened

That might be useful for some tools; but it has similar issues like
/proc/*/mem I think. But then, Linux also needs to do strange things
here for fd:s not corresponding to any filesystem location (like
pipes), so maybe that's not a problem...

>     I would like to have the suggestions of all you people in working
>     out what features are needed and what are really not necessary for
>     Hurd.

It's part of your task to find this out. As stated in the project
description, the goal is to make the procps and psmisc tools, and
various implementations of top or similar tools work. You need to find
out which features are necessary to make that happen.

>      I am used to #hurd IRC. I will be there most of the time I am
>      online. And also I have spoken to most of the guys there.

I'm looking forward to meeting you there, now that I'm online again :-)

>      Can any of you please give me the contacts of the authors of
>      procfs which currently exists in the hurdextras repo viz.
>      Jonathan Arney and James Morrison, so I can discuss with those
>      people too.

Those people haven't been active in Hurd development for a long time; we
don't have any contact. You could probably google up their current
addresses -- but they might not remember much of that stuff, or might
not be interested in discussing it... Don't expect too much from that
quarter.

-antrik-