[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Reducing iconv-induced memory usage
From: |
Ludovic Courtès |
Subject: |
Reducing iconv-induced memory usage |
Date: |
Tue, 26 Apr 2011 23:10:32 +0200 |
User-agent: |
Gnus/5.110015 (No Gnus v0.15) Emacs/23.3 (gnu/linux) |
Hello!
As Andy noted in the past, iconv conversion descriptors associated with
ports take up a lot of malloc’d memory, that only gets freed when
finalizers are run. On GNU/Linux, a UTF-8 → UTF-8 C.D., which does
nothing, mallocs 180 KiB (!), according to the program attached. So the
problem is acute.
So I think we should special-case UTF-8 I/O to not use iconv at all.
For output, it’s easy since we already do the conversion to UTF-8 in
‘display_string’. For input, it’s a bit more work because input byte
streams have to be checked for invalid sequences.
I’m working on a patch but I’d like to get initial feedback and also
about whether it should wait until after 2.0.1 or not.
Thanks,
Ludo’.
#include <iconv.h>
#include <malloc.h>
static size_t total;
static void * (*prev_hook) (size_t, const void *);
static void *
m (size_t s, const void *c)
{
__malloc_hook = prev_hook;
printf ("alloc %zi\n", s);
void *r = malloc (s);
total += s;
__malloc_hook = &m;
return r;
}
static void
my_init_hook (void)
{
prev_hook = __malloc_hook;
__malloc_hook = &m;
}
void (*__malloc_initialize_hook) (void) = my_init_hook;
int
main (int argc, char *argv[])
{
total = 0;
iconv_open ("UTF-8", "UTF-8");
printf ("allocated %zi B\n", total);
return 0;
}
- Reducing iconv-induced memory usage,
Ludovic Courtès <=