[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Varnamproject-discuss] Issues

From: Kevin Martin
Subject: Re: [Varnamproject-discuss] Issues
Date: Sun, 17 Aug 2014 14:45:42 +0530

Fixed. The issue was that strbuf old_ending that was added to the cache is changed at a later time. Fixed by creating a new strbuf val_buf, copying new_ending to val_buf and providing callback to strbuf_destroy.

On Sun, Aug 17, 2014 at 2:00 PM, Kevin Martin <address@hidden> wrote:
There is no specific error. It seems like some sort of memory corruption. But valgrind did not show anything. I could not find anything wrong with the cache logic.

Have a look at the screenshot attached. After sequentially typing words till കാരണമായ, I got കാരണസമൂ as a suggestion when I was trying to type കാരണം. I saw similar behavior when I was debugging. lru_cache_find() at times returned the input string rather than the stem rule. For example, at a stage during the stemming of അവിടെയാണെന്ന്, lru_find_cache returned അവിടെ as the value for a key ന്. I'm sure that lru_add_to_cache had NOT been called with അവിടെ as the key prior to this.

On Sun, Aug 17, 2014 at 9:06 AM, Navaneeth K N <address@hidden> wrote:
Hash: SHA512

Hello Kevin,

What is the error that you are getting? Did you debug it? If cache is
not throwing any error, it might be a issue with the caching logic
that you use. I can't see any obvious issues when I had a glance at it.

On 8/17/14 1:19 AM, Kevin Martin wrote:
> Solved most of the new issues:
>  Please take a look at lru_cache function inside vst_get_stem(). As
> of now they are faulty, because varnam does not stem properly with
> the cache. Once I comment out the cache, everything is fine. I'll
> resolve this today (17th Aug) but I'm almost clueless. Are there
> any obvious mistakes I'm making with lru_cache?
> I ran "valgrind ./stemmer stemmer_input.txt" on the example and it
> did not show anything other than 2 "Conditional jump or move
> depends on uninitialised value(s)". There are other memory leaks. I
> will fix those too today.
> On Fri, Aug 15, 2014 at 3:56 PM, Kevin Martin
> <address@hidden> wrote:
>> Solved all issues except #64, #69, #56 and #55
>> #69 - stem() should not be util.h. No idea where to put stem().
>> The only downside of not putting it in a header file is that the
>> tests won't work. If we do not have direct access to stem(), we
>> will have to call varnam_learn() and then see if the stemmed word
>> was learned (by querying the db) to test the stemmer
>> functionality.
>> #64 - make get_stem() more efficient. The prepared statements
>> have been stored in handle->internal. Still figuring out how to
>> use the lru cache and where.
>> #55 - I used to run valgring on examples/stemmer.c. Since that
>> example is now removed from the project, how should I run the
>> valgrind tests? An option is to run it on examples/learning since
>> varnam_learn() calls stem().
>> #56 - integrate stemmer.

- --
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: GPGTools -


reply via email to

[Prev in Thread] Current Thread [Next in Thread]