[silpa-discuss] Re: Optimization idea

[Santhosh Thottingal]

On Sat, Apr 24, 2010 at 8:16 PM, Vasudev Kamath <address@hidden> wrote:
> Hi,
> Idea looks good but i've some doubts here.
> 1. Index file is it based on first letters or language. What i meant is we
> will have a single file with words for all languages in it and index file
> contains starting line number for each languages or is it for each letter in
> each language?.

Separate dictionaries for each languages. Why I don't want to touch
the existing dictionary is , these dictionaries are coming from
various upstreams projects , mainly hunspell_xx_IN dictionaries, where
xx_IN is any Indian languages.

> 2. I looked at the linecache module, we have getline function which allows
> us  to get a particualar line specifed by line number. So we may need to
> loop to read all the words incrementing the line number

Yeah, I just now looked into the linecache modules code, It is just
wrapper over getlines. But instead of using linecache,
file.seek(position) should not have this limitation.

> 3. How we will know where the words starting with a letter comes to end. i.e
> is it sufficient only to have starting line number in the index file or do
> we need ending line number?. There is one way to calculate ending line
> number since index file contains next letter's starting index we can stop 1
> line before that number but how about the processing complexity or
> implementation strategy?.

We need to know only start position. we can continue reading lines
till the first letter changes.


Thanks
Santhosh