Re: [Logs-devel] LoGS userfriendliness?

[Jim Prewett]

Hi Vijay,

This all looks very nice!  Good work!  Now, let me rip it apart (I hope 
you will return the favor soon :)

> Also, since speed is the key, it's better we directly implement the
> macro rather than have it generated.

I don't understand what you mean; I'm having problems parsing that 
sentence.

> (LoGS with messages from file filename
>      with rule matching "message" and length > 10
>      with rule matching regexp "^.*success\\." and rule matching "user"
>      do magic)
> 
> would generate
> 
> (with-open-file (#:messages filename)
>  (let ((#:rule1 (make-instance 'rule :match (lambda (#:msg)
>                                              (string= (message #:msg)
>                                                       "message")))n)
>       (#:rule2 (make-instance 'rule
>                               :match (lambda (#:msg2)
>                                        (cl-ppcre::scan "^.*success\\."
>                                                        (message #:msg2)))
>                               :action (make-instance 'rule
>                                               :match (lambda (#:msg3)
>                                                        (string= #:msg3
> "user")))))
>       ((#:ruleset (make-instance 'ruleset))))
>    (enqueue #:ruleset #:rule1 #:rule2)
>    magic))                            ; this needs to be done
> 
> please point out impracticalities and other mistakes of all magnitudes.


In LoGS, there are these objects called 'File-Follower that essentially 
allow us to do something like 'tail -f' on a logfile.  File-Followers also 
handle file rollovers (syslog regularly moves /var/log/messages to 
/var/log/messages.1 and then opens a new /var/log/messages).  In theory, 
LoGS could watch an infinite stream of these files.  with-open-file is 
very nice in terms of clarity, but... 

I'm not sure that I like the infix notation in length > 10. (Ha! 
non-lispers, I can play that game too! ;)

The hard part, IIRC, is when you're trying to generate new rules. :)

Lets say I'm doing some security analysis work.  I set up some (likely 
complex) scheme to detect port scans from IP addresses.  When I see such a 
port scan, I want to watch to make sure that I don't get any successful 
user logins from that host.  When I do see a login from one of those hosts 
for username fred, I want to gather every message I can find that mentions 
fred, or the IP.  (oooh!  I think thats a *nice* example!  I'm going to 
find a paper or presentation to stick that in! :)

That can probably be accomplished in some way with all of LoGS, 
Logsurfer, and SEC.

> we need to include words for contexts and ways to delete rules.

I think you're forgetting about the fact that rules have (gobally unique 
and stored in a hash table by them) names.  If a name is not specified, a 
symbol is generated (gensym) to be used for the name.

My idea there was that rules could be easily retrieved for deletion by 
name, even if one was not supplied by the user.

> a rough grammar would be
> 
> with messages from {[file filename]|[stream streamname]}
> {with rule <rule-syntax>}+
> <rule-syntax> := matching [regexp] message and <rule-syntax-clauses>
> 
> essentially, with rule means create a new rule and, "and rule" implies
> generate a new rule.  the part with "length > 10" would be (method op
> form) where method would expand as (lambda (#:msg) (op (method msg)
> form)).  This way, adding new classes of message could have other
> methods called.  We probably need to say
> 
> perform method op form so that method can be anything.

ok.  that sounds good. 

> i'm just throwing ideas.  will do concrete work over the week whenever
> i can squeeze it in.

> i'll download the latest source tomorrow :-)

ttys,
Jim

p.s. *DAVE*, join this mailing list!!!