Re: How to make GNU Guile more successful

[Nala Ginrut]

s/Many we could find a way/Maybe we could find a way
Sorry

On Sat, Jul 15, 2017 at 8:55 PM, Nala Ginrut <address@hidden> wrote:
> @Linas Thanks for sharing your experiences on large data processing
> using Guile! I'm developing a framework to do the similar work for our
> product too. But what I'm planning is to do map/reduce for large data
> in a distributed system. Anyway, I don't think it's a good way to
> analysis massive data in just one very-strong machine. It requires too
> much for the language compiler.
>
> I'm still very confident to use Guile on machine learning work. Please
> don't forget that no any dynamic language do the heavy computation
> directly, they wrapped BLAS or MKL, just like what numpy does. And I
> have chance to get a very strong Nvidia GPU cluster server for the
> work on the next stage. I don't think the computing speed will be the
> issue for me. We're using Python and C++14 at present, but I don't
> think Python is the idea glue language, no one wants to use Cython,
> trust me. The tail-recursive and delimited-continuation for
> lightweight task scheduling is preferred too. The key issue for me is
> the expressiveness, which could let us type less code to do more work.
> Actually, we use Guile to generate many C++ code in compiling time to
> save lot work coding work. This does matter, especially when you have
> few people but much work to finish in a tight deadline.
>
> There should be better paradigm to use Guile to glue C/C++ code.
> Implement algorithm in Guile directly is not the good way. And wrap
> C++ interface with FFI simply is not good too. Sometimes we need to do
> more abstract work to get both advantages of Guile and C/C++.
> This is about how we use Guile, not the compiler issue.
>
> Of course, I confess there're many problems in our current Guile
> compiler. The problem is that we have to use Guile a lot to get to
> know what is the problem exactly, and give compelling reasons for the
> improvement.
>
> @Jan Yes, that should be a way to go. And I have a new idea which is
> just an idea at present. Many we could find a way to read PyObject to
> Gulie, and call Python module directly (say, numpy). There should be a
> type-compatible abstract level between Guile and PyObject. If it
> works, we may implement Python3 on Guile. Although it seems a large
> work to implement complete Python3 frontend, we may save lot of work
> to write alternative Python modules for Guile.
> Julia language does in this idea, but it's backend is compatible with
> Python. My idea is not to convert all Python types to Guile, just wrap
> some types to a special object like <pyobject> is enough, then Guile
> could be glue language for Python too.
>
> Maybe people ask, why bother to glue Python? Python rocks!
> Rocks? I need complete lambda calculus, proper tail call, static
> scoping, and delimited-continuations (not the generators).
> In our product code, we use lot of lambdas in C++ code, so I expect
> the programming mind could be consistent between glue language and
> main language. But Python can't provide multi-lines lambda expression
> only because it uses whitespaces for delimiters, there's no any chance
> to delimit multi expression within a lambda unless introducing more
> complexity to the parser.
>
> Back to the topic, Guile lacks sufficient practical usage in industry
> to reveal its disadvantages. But its advantages are apparent enough
> for me so that it worth for me to go further with it.
>
>
> Best regards.
>
>
>
>
> On Sat, Jul 15, 2017 at 6:10 PM, Jan Wedekind <address@hidden> wrote:
>> One could implement something like Theano+NumPy with GNU Guile. I am trying 
>> to do something like that (github.com/wedesoft/aiscm) but I am doing it in 
>> my spare time only. Theoretically GNU Guile is better suited for this than 
>> Python because of macros.
>>
>> On July 14, 2017 10:54:45 PM GMT+01:00, Linas Vepstas <address@hidden> wrote:
>>>On Mon, Feb 13, 2017 at 2:28 PM, Panicz Maciej Godek
>>><address@hidden
>>>> wrote:
>>>
>>>>
>>>> someone
>>>> responded critically: "are there out of the box libraries to estimate
>>>a
>>>> zero inflated negative
>>>> binomial regression model in guile". Of course, if I knew what a
>>>> zero-inflated
>>>> negative binomial regression model, I could deliver an implementation
>>>by
>>>> just explaining
>>>> the notions used in that phrase.
>>>
>>>
>>>Caution: the message below sounds negative.  Sorry, I use guile daily
>>>and
>>>almost exclusively now. So there ...
>>>
>>>Lack of decent science libraries for scheme is a major stumbling block,
>>>for
>>>me. Simply having sine and cosine is not enough.   I got excited (a
>>>decade
>>>ago) when I realized that guile supported GnuMP, and then rapidly
>>>deflated
>>>when I realized it only supported integers and rationals in GnuMP .. I
>>>work
>>>with arbitrary-precision floats.  Or, I did back then.
>>>
>>>Maybe more important is making guile work well with large-RAM setups.
>>>Currently, I do data analysis, every day, in guile, on datasets that
>>>take
>>>20GB or 40GB -- my current one is 110GB when loaded in RAM, and guile
>>>starts getting buggy, crashy and slow when working at that size.
>>>Sometimes, it starts calling GC half-a-dozen times per second, for no
>>>apparent reason, eating up 6 cores (or more!) doing nothing but GC.
>>>Why?
>>>Who knows? Who can tell?
>>>
>>>Yes, I have a machine with 256 GB RAM and a few dozen cores, and SSD's
>>>that
>>>hold the data, but every time guile crashes, I have to wait an hour for
>>>the
>>>data to reload.  I can live with it, but its a dirty secret I would not
>>>share with guile wannabe users.
>>>
>>>String handling in guile is a disaster area: If I give it a
>>>10-megabyte-long string in utf8, it promptly tries to convert all of
>>>that
>>>string in utf32, for utterly pointless reasons. This just makes it
>>>slow.
>>>
>>>There are still bugs between GC and the compiler: if call (eval "(some
>>>stuff) (other stuff)")  the compiler will try to compile that string
>>>(after
>>>it was converted ti utf32!) and if GC happens to run at just that
>>>moment,
>>>guile crashes or hangs.  These bugs need to be fixed.
>>>
>>>So although its a good start, there's a lot of work left until it can
>>>get
>>>to "the next level". And that work can't happen until guile is more
>>>popular. So it's very much chicken-and-egg scenario.
>>>
>>>--linas
>>
>> --
>> Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.