[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[potluck dish] crawling the Freenet WoT with sxml and match

From: Arne Babenhauserheide
Subject: [potluck dish] crawling the Freenet WoT with sxml and match
Date: Tue, 16 Feb 2016 23:37:41 +0100
User-agent: mu4e 0.9.15; emacs 24.5.1

Hi Guilers,

I’d like to contribute a small dish, I’ve been working on during the
past two weeks.

A first script downloads xml files of anonymous identities in Freenet,
extracts trust given to other identities and downloads these, too. A
second script parses the identities and turns them into a csv file which
social network researchers can use to investigate the effects of
anonymity on social interaction (at least that’s the goal). A third and
fourth script anonymize and deduplicate the results.

What I like best is using match and sxml to extract the relevant
identifiers, though I’m using mutation to separate the match in
the let-recursion and a final cons which saves the filename:

    (define (parse-trust-values filename)
      (let* ((port (open-input-file filename))
             (sxml (non-breaking-sxml-reader port))
             (closed (close-port port))
             (trust '()))
        (let extract-trust ((sxml sxml))
          (match sxml
            (('Trust ('@ ('Value value) ('Identity uri) rest ...))
             (set! trust
               (cons (cons (wot-uri-key uri)
                           (string->number value))
            ((a b ...)
             (map extract-trust sxml))
            (else '())))
        (cons (wot-file-key filename) trust)))

I attached a network visualization generated from the data. Some
explanation is available from my website:

Attachment: trust-sone-reingold.png
Description: Sone social graph in Reingold Layout

The code is available in the guile-freenet repository on and on Bitbucket and from a static clone:

Happy Hacking!
Arne Babenhauserheide

PS: I wrote this code without wisp to make it possible to run it with
    Guile out-of-the-box. To be suitable for programs I share, I’d need
    either guildhall integrated into Guile (to provide wisp with just a
    single command) or wisp itself in Guile.
Unpolitisch sein
heißt politisch sein
ohne es zu merken

Attachment: signature.asc
Description: PGP signature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]