bug-cvs
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [bug-gnulib] valloc()?


From: Derek Price
Subject: Re: [bug-gnulib] valloc()?
Date: Wed, 02 Mar 2005 11:49:59 -0500
User-agent: Mozilla Thunderbird 1.0 (Windows/20041206)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I asked Jim Kingdon, who I thought I might be able to blame for the
original code, why valloc() was chosen.  He replied:

| Well, if the buffer.c buffers are split across pages, you might
| have two page faults instead of one, the kernel would have to
| maintain two page table entries instead of one, etc.  So in that
| sense, page-aligned is the right thing, but not at all a big deal
| since we are allocating 4096*16 bytes at a time.
|
| Some networking kernels can probably avoid a copy of the data if
| you pass a page-aligned buffer to system calls like
| recv/send/read/write, which might be a bigger performance win.
|
| Now, many versions of malloc will page-align an allocation that big
|  without us having to ask for it.
|
| The verdict?  valloc (or its relatives like posix_memalign) isn't
| worth the portability hassles.  Although I suppose if you have
| doubts you could try to come up with some suitable benchmark (you'd
| have to figure out what would hit buffer.c hard enough, and find a
| malloc which would misalign pages for you).
|
| (None of this is based on my memory of why that code got into CVS -
|  presumably because Ian wrote it that way and no one thought to
| advocate changing it).


Based on this, I might choose to implement your suggestion, Bruno (and
change the default buffer data size in CVS to the system page size):

| This is the best you can do in a portable way, but 1. the return
| value cannot be passed to free(), 2. it wastes 1/2 page of memory
| on average.
|
| Therefore I'd suggest a new interface: void*
| pagealign_alloc(size_t); void pagealign_free(void*);
|
| and do the implementation as follows: - If mmap() is available, use
| mmap and some bookkeeping for pagealign_alloc, and munmap() for
| pagealign_free, - Otherwise, if posix_memalign() is available, use
| it and free(), - Otherwise, use something similar to the valloc()
| above.


But I have a few questions.  Why mmap?  What sort of object would you
suggest I map the allocation to?  My Linux man page says that the
MAP_ANONYMOUS flag (which requires no object to map) has been
supported since the 2.4 kernel, but I can find no reference to
MAP_ANONYMOUS in the POSIX spec.

(Most of this is hardly necessary for CVS since it is already caching
unused buffer datas rather than freeing them, and reusing them as they
are needed.  All we really need is the valloc() module, though it
might be nice to default to posix_memalign() (or mmap()?) instead,
when one can be found, simply to avoid the wasted space before the page.)

Regards,

Derek


Derek Price wrote:

| Bruno Haible wrote:
|
| | Is this all worth it? For what purpose do you need the memory to
| be |  page-aligned?
|
|
| That's a good question, as the original isn't my code.  I was just
| assuming that whoever wrote it originally knew what they were
| doing.
|
| There is only one place that CVS does this, and it is where it is
| allocating new, empty buffer datas.  It later uses theres primarily
| in calls to read() from various sources, from network pipes, exec
| pipes, to files.
|
| Looking at it, it mitigates the average 1/2 page lossage by
| allocating 16 * 4096 byte buffers at once, but this also means that
| it is not making any attempt to force later buffer datas to be page
| aligned - if this is compiled on a system with a page size greater
| than 4096, some of the buffer datas will not be page aligned.  Is
| there any good reason for doing this?  The buffers are saved and
| reused, and their ordering is not preserved, so I wouldn't expect a
| great performance advantage from doing this.
|
| Regards,
|
| Derek
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (Cygwin)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCJe63LD1OTBfyMaQRAr9GAKCSMQzfkcaOs6pwagc9e1uqk4jwlACgz7ym
fL9XWHl5cHBVkxfHhicBFT4=
=W5eY
-----END PGP SIGNATURE-----






reply via email to

[Prev in Thread] Current Thread [Next in Thread]