Re: [Varnamproject-discuss] Issues

[Kevin Martin]

There is no specific error. It seems like some sort of memory corruption. But valgrind did not show anything. I could not find anything wrong with the cache logic.

Have a look at the screenshot attached. After sequentially typing words till കാരണമായ, I got കാരണസമൂ as a suggestion when I was trying to type കാരണം. I saw similar behavior when I was debugging. lru_cache_find() at times returned the input string rather than the stem rule. For example, at a stage during the stemming of അവിടെയാണെന്ന്, lru_find_cache returned അവിടെ as the value for a key ന്. I'm sure that lru_add_to_cache had NOT been called with അവിടെ as the key prior to this.

On Sun, Aug 17, 2014 at 9:06 AM, Navaneeth K N <address@hidden> wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hello Kevin,

What is the error that you are getting? Did you debug it? If cache is
not throwing any error, it might be a issue with the caching logic
that you use. I can't see any obvious issues when I had a glance at it.

On 8/17/14 1:19 AM, Kevin Martin wrote:
> Solved most of the new issues:
>
> https://github.com/lonesword/libvarnam/commit/dfbd710c0f31b41d0d0e4d9f49444a65425ba0b1
>
>  Please take a look at lru_cache function inside vst_get_stem(). As
> of now they are faulty, because varnam does not stem properly with
> the cache. Once I comment out the cache, everything is fine. I'll
> resolve this today (17th Aug) but I'm almost clueless. Are there
> any obvious mistakes I'm making with lru_cache?
>
> I ran "valgrind ./stemmer stemmer_input.txt" on the example and it
> did not show anything other than 2 "Conditional jump or move
> depends on uninitialised value(s)". There are other memory leaks. I
> will fix those too today.
>
>
> On Fri, Aug 15, 2014 at 3:56 PM, Kevin Martin
> <address@hidden> wrote:
>
>> Solved all issues except #64, #69, #56 and #55
>>
>> #69 - stem() should not be util.h. No idea where to put stem().
>> The only downside of not putting it in a header file is that the
>> tests won't work. If we do not have direct access to stem(), we
>> will have to call varnam_learn() and then see if the stemmed word
>> was learned (by querying the db) to test the stemmer
>> functionality.
>>
>> #64 - make get_stem() more efficient. The prepared statements
>> have been stored in handle->internal. Still figuring out how to
>> use the lru cache and where.
>>
>> #55 - I used to run valgring on examples/stemmer.c. Since that
>> example is now removed from the project, how should I run the
>> valgrind tests? An option is to run it on examples/learning since
>> varnam_learn() calls stem().
>>
>> #56 - integrate stemmer.
>>
>

- --
Cheers,
Navaneeth
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: GPGTools - https://gpgtools.org

iQEcBAEBCgAGBQJT8CMrAAoJEHFACYSL7h6ky1oIAIOeZe+Q5lp63+hUIej9lDbp
RTprHNiarwpbVtZM9Imfxj79bmPiqkHayLg2r3dWXw1W7+sYOG58JNcANuWDpAkj
d0D6O6FqaH12RQ0+Vkxcv6h0xOTY1BWs1m/hFUMKUbYRHG1RGwXwJcn8RXkSmdgc
6WDUGeSGustcLzt7UmkA+z5xf/seIktr25S5iFd+RsAeiQnB04oETCIv3hqbTYVl
kTESgAEmYypA91sctnUsSK39Xu9dR60lQwp4PGv4XZMVQcnPNiqv2N1XdhML/U8x
V0qYofwl3TO3wEWHgkJfqQKUCwVBubTgWgjfi0otbv5Pwsz9tbF3IZnCFzeJV60=
=E+nZ
-----END PGP SIGNATURE-----

problem.png
Description: PNG image