[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: babelia

From: Amirouche Boubekki
Subject: Re: babelia
Date: Fri, 6 Dec 2019 14:19:59 +0100

Hello all!

Le sam. 16 nov. 2019 à 11:06, Amirouche Boubekki <
address@hidden> a écrit :

> I restarted working on my personal search engine.

After two weeks of work, 41 files changed, 1845 insertions(+), 441
deletions(-) and 97 commits, I tagged a v0.2.0 in the repository at:

The babelia index and babelia search subcommands were removed. Instead, one
has to `make web` to spawn a server and then hit the
/api/search?query=foobar to make a search. To index stuff, one can POST a
file like test.scm to /api/index or rely on babelia crawler subcommands.
The crawler is still a work-in-progress. Do not expect the index to be
compatible with future releases.

> The last iteration, gotofish, was not too bad even if it has bitrot.
> Based on my research and practical experiment, it seems very clear
> that there is no workaround the use of map-reduce, that might be known
> as n-par-for-each [3].
> [3]
> I made a prototype similar to that n-par-for-each, except it works
> with guile-fibers, is asynchronous and works with a shared pool of
> threads instead of spawning N threads for each incoming query like
> gotofish does.

Actually, what I need is n-for-each-par-map where map happens in parallel.
The implementation can be found in the babelia/pool.scm file [4].


The installation process is still a little bit akward, because one needs to
change the path to wiredtiger-3.2.0-0 shared library in the source. Add my
channel [5] and do `make init` to get started.


Happy hacking!

reply via email to

[Prev in Thread] Current Thread [Next in Thread]