libidn2 code size

[Simon Josefsson]

All,

IDNA implementations are unfortunately quite big due to the various
tables needed in memory, so I wanted to compare libidn and libidn2.  I
built libidn 1.20 and libidn2 0.1 on my Debian squeeze amd64 machine
with CFLAGS=-Os and sorted all objects based on size and the lists are
shown below.  Libidn is ~237Kib and libidn2 is ~298Kib. :-(

I have good hopes to shrink libidn2 somewhat though, considering:

1) scripts.o.  IDNA2008 only needs to identify Greek (511), Hebrew
(133), Hiragana (91), Katakana (300), and Han (75960) code points.  A
specific table for them ought to be possible to achieve in less size
than scripts.o suggests below -- that file contains information about
all scripts.

2) I don't have high hopes for decreasing the NFC implementation size
considerably, that is known to take some space.

3) data.c is inefficiently stored (and searched) right now, it should
use gperf instead.

4) researching whether all abilities of categ_of.o and bidi_of.o is
needed or if some parts of it can be short-circuited.

5) Allow building libidn2 without libiconv stuff?  That should reduce
size somewhat.

Thoughts and ideas on this are most welcome!

/Simon

address@hidden:~/src/libidn2-0.1$ LANG=C ls -lSa `find . -name .libs`|grep 
^-rw-|sed 's/-rw-r--r-- 1 jas jas *//g'|sort -g -r
298854 Mar 30 00:15 libidn2.a
241778 Mar 30 00:15 libgnu.a
47816 Mar 30 00:15 scripts.o
46219 Mar 30 00:15 decomposition-table.o
28915 Mar 30 00:15 data.o
21920 Mar 30 00:15 composition.o
21048 Mar 30 00:15 categ_of.o
14944 Mar 30 00:15 bidi_of.o
11184 Mar 30 00:15 striconveh.o
10392 Mar 30 00:15 combiningclass.o
6392 Mar 30 00:15 error.o
5344 Mar 30 00:15 striconveha.o
3872 Mar 30 00:15 categ_M.o
3752 Mar 30 00:15 localcharset.o
3504 Mar 30 00:15 idna.o
3392 Mar 30 00:15 u32-normalize.o
3368 Mar 30 00:15 register.o
3368 Mar 30 00:15 context.o
3080 Mar 30 00:15 joiningtype_of.o
2952 Mar 30 00:15 punycode.o
2792 Mar 30 00:15 lookup.o
2704 Mar 30 00:15 u8-conv-from-enc.o
2576 Mar 30 00:15 c-ctype.o
2408 Mar 30 00:15 decompose-internal.o
2320 Mar 30 00:15 u32-to-u8.o
2184 Mar 30 00:15 u8-to-u32.o
2000 Mar 30 00:15 bidi.o
1920 Mar 30 00:15 tables.o
1840 Mar 30 00:15 canonical-decomposition.o
1744 Mar 30 00:15 malloca.o
1728 Mar 30 00:15 u8-strconv-from-enc.o
1624 Mar 30 00:15 u8-mbtouc-aux.o
1608 Mar 30 00:15 version.o
1552 Mar 30 00:15 u8-mbtoucr.o
1528 Mar 30 00:15 c-strncasecmp.o
1520 Mar 30 00:15 u8-strconv-from-locale.o
1496 Mar 30 00:15 c-strcasecmp.o
1488 Mar 30 00:15 u8-mbtouc-unsafe-aux.o
1480 Mar 30 00:15 categ_none.o
1440 Mar 30 00:15 u8-check.o
1400 Mar 30 00:15 u8-prev.o
1376 Mar 30 00:15 free.o
1368 Mar 30 00:15 u8-uctomb-aux.o
1360 Mar 30 00:15 nfc.o
1320 Mar 30 00:15 categ_test.o
1272 Mar 30 00:15 u8-mblen.o
1256 Mar 30 00:15 nfd.o
1216 Mar 30 00:15 u8-strlen.o
1216 Mar 30 00:15 u32-cpy.o
933 Mar 30 00:15 u32-mbtouc-unsafe.o
932 Mar 30 00:15 u8-mbtouc-unsafe.o
926 Mar 30 00:15 u32-uctomb.o
925 Mar 30 00:15 u8-uctomb.o
925 Mar 30 00:15 u8-mbtouc.o
918 Mar 30 00:15 libidn2.lai
address@hidden:~/src/libidn2-0.1$ 

address@hidden:~/src/libidn-1.20/lib$ LANG=C ls -lSa `find . -name .libs`|grep 
^-rw-|sed 's/-rw-r--r-- 1 jas jas *//g'|sort -g -r
236974 Mar 30 00:22 libidn.a
91552 Mar 30 00:22 nfkc.o
76858 Mar 30 00:22 rfc3454.o
10672 Mar 30 00:22 profiles.o
10318 Mar 30 00:22 libgnu.a
7512 Mar 30 00:22 pr29.o
7200 Mar 30 00:22 idna.o
4816 Mar 30 00:22 stringprep.o
4112 Mar 30 00:22 striconv.o
4032 Mar 30 00:22 tld.o
3136 Mar 30 00:22 strerror-stringprep.o
3080 Mar 30 00:22 strerror-idna.o
2936 Mar 30 00:22 punycode.o
2752 Mar 30 00:22 strerror-tld.o
2576 Mar 30 00:22 c-ctype.o
2320 Mar 30 00:22 toutf8.o
2208 Mar 30 00:22 strerror-punycode.o
2144 Mar 30 00:22 tlds.o
2120 Mar 30 00:22 strerror-pr29.o
1616 Mar 30 00:22 version.o
1528 Mar 30 00:22 c-strncasecmp.o
1496 Mar 30 00:22 c-strcasecmp.o
1376 Mar 30 00:22 idn-free.o
915 Mar 30 00:22 libidn.lai
address@hidden:~/src/libidn-1.20/lib$ 

/Simon