Re: [ANN] guile-msgpack: MessagePack for GNU Guile (+ help needed)

[Thompson, David]

On Wed, Sep 19, 2018 at 8:34 AM HiPhish <address@hidden> wrote:
>
> ## What is MessagePack ##
>
> MessagePack is a specification for how to turn data into byte and vice-versa.
> Think of it like JSON, except instead of human readability it stresses speed
> and size efficiency. The format is binary and well suited for data exchange
> between processes. It also forms the basis of the MessagePack RPC protocol[2].

This is cool! I've seen people use MessagePack for multiplayer video
games.  Maybe now that this library exists I will give that a try some
day.

> ## About the Guile implementation ##
>
> My implementation is written in pure Guile Scheme, so it should work out of
> the box. You can try it out without installing anything. Just clone the repo
> and give it a try:
>
>     $ guile -L .
>
>     scheme@(guile-user)> ,use (msgpack)
>     scheme@(guile-user)> (pack "hello" 1337 #t)
>     $1 = #vu8(165 104 101 108 108 111 205 5 57 195)
>
> As you can see, the three objects passed to the `pack` procedure have been
> turned into one big bytevector. We could now unpack this bytevector again,
> write it to a file, or send if off to a port. Since using MessagePack with
> ports is a frequent task there is also a `pack!` procedure which takes in a
> port to pack to as well.
>
>     (define hodgepodge (vector 1 2 '#(3 #t) "foo"))
>     (pack! hodgepodge (current-output-port))
>
> To get our object back we `unpack` it:
>
>     (unpack #vu8(#xA5 #x68 #x65 #x6C #x6C #x6F))  ; Returns "hello"
>     (unpack! (current-input-port))

So does this allow for sending over arbitrarily nested s-expressions
(i.e. anything supported by Guile's 'read' and 'write' procedures)?
For example:

    (pack '((foo . 1) (bar . (2 3 4)) (baz . "hello")))

> How much is portability a concern? I know Guile implements *some* or r6rs, but
> I wasn't paying much attention to that. Is it something worth considering or
> should I just treat Guile as its own language that just so happens to be based
> on Scheme?

FWIW I do the latter. I write my projects specifically for Guile, but
often my projects have to be that way because there's no Scheme
standard that provides the interfaces I use.

> The extension type `ext` (msgpack/ext.scm) is a pair of a signed 8-bit integer
> and a bytevector. The constructor does not enforce this condition, the two
> slots can be really anything. What it the proper way of enforcing this in
> Guile? I know Common Lisp has type declarations and Racket has contracts, but
> what does Guile have?

Guile doesn't provide any static type system. I haven't looked at your
source, but when you say "a pair", do you mean a literal cons cell?
If so, I recommend switching to record types via the (srfi srfi-9)
module.  You can write a custom constructor that does the type checks
if you'd like.

> The `pack` procedure takes any number of objects and packs them into a large
> bytevector. However, the `unpack` procedure only returns the first object it
> unpacks. Is there a way of making it unpack as many as it can? I thought of
> `values`, but you would need to know the number of values in advance. Also the
> caller would have to know in advance how many objects he is going to get
> unpacked.

You could resolve this by changing 'pack' to only accept a single
value. If the user wants to pack multiple items, then they would pass
a list or vector.

Hope this helps,

- Dave