Re: centralized monitoring

[Jan-Henrik Haukeland]

Georges Toth <address@hidden> writes:

>> It's a good idea, which we have discussed before. In fact I'm about
>> to design and write such an application now.
>
> oh, ok :-)
> is it possible to have a look at your design please ?

Sure, but at the moment the design in mostly inside my head. Enclosed
is a first stab at the design. The drawing needs some explanations:

The setup is as follows: monit is running on several servers (the blue
boxes). We also assume that the monit enabled servers are running
inside a demilitarized zone. (We assume this because it's a common
setup. This DMZ is not necessary for the design, but will explain the
design choices I made).

The centralized monit application (lets call this application XZZY
until I can find a better name) is implemented as C Servlets, using
the zild application server. This application server is implemented by
me and I have attached a zild paper with this mail for your
information. (Note, not every listed feature are available in the
current zild beta). The purpose of using zild is twofold, first, it is
a very good framework for implementing the XZZY application and
second, I want to develop a useful and interesting application using
zild (to showcase zild).

In the first version, the XZZY application will provide two main
functionalities;

1.0) From one XZZY web page, it should be possible to view all monit
     instances and their status. It should also be possible to click
     on one host icon and access the monit HTTP status page on a
     certain machine as well as start, stop, restart and unmonitor
     applications on the same machine using the local monit
     instance. This is illustrated with the green servlet in the
     drawing.

1.1) In a later version, it should also be possible to access each
     monit instance from XZZY and configure the monit instance over
     the web, thus, instead of writing a monitrc control file for each
     monit instance, a monit instance can be configured from XZZY's
     web page.

2.0) The second functionality is about history. Historical data is
     important in SLA (Service Level Agreement) contracts and can be
     used to document server uptime and what and where something went
     wrong. This functionality is illustrated with the orange servlet.

     Here, I first thought about the following simple solution; When
     an error occurs on a host, monit will in addition to sending an
     email alert, use HTTP POST and send the problem description to
     the XZZY application. Zild has it's own embedded SQL database
     (http://www.sqlite.org) and the XZZY application will save
     problem descriptions to the database. There will also exist web
     pages in XZZY where you can view historical data, uptime and such
     for each monit box.

     But after speaking with a couple of sys.adm. friends about this
     design, they pointed out that this will require that yet one more
     port it opened in the Firewall between the Secure zone and
     demilitarized zone, which is always a problem, since a sys. adm.
     will want very few ports opened between these two zones (if
     any). The solution we came up with is simple, monit will send
     alert emails to a SMTP server as usual, but to a special
     account. The XZZY application will regularly poll the email
     server and download alerts and store them into the database as
     mentioned above. This way, no new open ports are necessary in the
     Firewall. (Email alerts should be formated in a special way so
     it's easy to parse emails and store the data in a database. Monit
     already provides the functionality to write email alerts on a
     certain format).

2.1) In addition to 1.1 above (in a later version) is to have each
     monit instance send an "up-email" to the XZZY email account upon
     start. This way it's not necessary to configure XXYZ with new
     monit instances; If a new email is received about a new
     monit/host instance, XZZY will simply show this in the web
     interface. The effect is that adding new monit/host instances is
     registered automatically in XZZY.


Note: The XZZY application will probably take me some time (1-2
months) to implement and I will most likely not give it away under the
GPL license. The license will probably be something like the Big
Brother license (http://bb4.com/license.html). This means that it's
free for non-commercial use, but you will have to pay a small fee if
it's used commercially.

centralmonit.jpg
Description: centralmonit.jpg

zild.pdf
Description: zild.pdf

-- 
Jan-Henrik Haukeland