Re: monit '-i' option - one new alternative and summary

[Martin]

Summary of currently known possible solutions:

1.) '-i' option
---------------

Usage:
- add following configuration to monitc (to start it in the case that it is not
  running):

    check process myprocess with pidfile /var/run/myprocess.pid
        start program = "/etc/init.d/myprocess start"
        stop program = "/etc/init.d/myprocess stop"
        alert address@hidden

Advantage:
- simple

Disadvantage:
- does not solve fundamental cause of the problem => the problem can still
  occure regardless whether the service is slow starting or failed to start
- the behavior is global for all monit services (sleep) - monit will not
  monitor regardless whether the service is running or not


2.) start slow or critical service from monit directly
------------------------------------------------------
Usage:
- disable the service startup in appropriate or all runlevels. You can use
  for example on RedHat:

    chkconfig myprocess off

  on Debian:

    update-rc.d -f myprocess remove

  general example:

    mv /etc/rc2.d/S99myprocess /etc/rc2.d/s99myprocess

- add following configuration to monitc:

  check process myprocess with pidfile /var/run/myprocess.pid
        start program = "/etc/init.d/myprocess start"
        stop program = "/etc/init.d/myprocess stop"
        alert address@hidden

Advantage:
- simple - the monit configuration is the same as in the case of -i option, you
  need only to disable the default startup
- solves the problem cleanly - no race condition can occure, the service is
  completely under monit's control (which is the case of -i option too - monit
  takes over the responsibility for service, but after it was started from rc)
- makes no sleep - services are started as fast as possible in the order as
  configured in monit (+ dependency support) => monit starts monitoring as soon
  as possible

Disadvantage:
- the user must handover/reconfigure the startup responsibility (as outlined 
above)


3.) wait for service to start in appropriate rc script
------------------------------------------------------
Usage:
- add following line to /etc/inittab behind rc:

    id:2:initdefault:
    ...
    l2:2:wait:/etc/init.d/rc 2
    ...
    mo:2345:respawn:/usr/local/bin/monit -Iic /etc/monitrc # added line

- add following configuration to monitc:

  check process myprocess with pidfile /var/run/myprocess.pid
        start program = "/etc/init.d/myprocess start"
        stop program = "/etc/init.d/myprocess stop"
        alert address@hidden

- wait in the rc script of the service to start or sleep in it some time (to
  pause the startup and hope that the service will be online in the meantime).

Advantage:
- per service - you need to care only for slow starting services

Disadvantage:
- it will pause the whole startup - init will not continue unless the service
  will be online or fail or the sleep will finish
- in the case that the startup failed, monit will detect it independently of
  rc script (doppled control) => the fundamental cause can affect this solution
  too in the case that the service failed to start


4.) enable the service monitoring manualy
-----------------------------------------
Usage:
- add following configuration to monitc:

    check file myprocess.pid with path /var/run/myprocess.pid
        if timestamp > 5 minutes then
           exec "/usr/bin/monit -c /etc/monitrc monitor myprocess"

    check process myprocess with pidfile /var/run/myprocess.pid
        start program = "/etc/init.d/myprocess start"
        stop program = "/etc/init.d/myprocess stop"
        alert address@hidden
        mode manual

  This way you can enable monitoring after specified time since process startup.
  There should be allways pidfile which can be monitored - in the case that
  myprocess is started from some sort of dispatcher and its pidfile will be
  created later then monit starts, you can monitor the pidfile of such
  dispatcher. In the case that this is not possible and there is no other
  suitable file for timestamp monitoring, you can touch some file in the slow
  service startup script (which is simple).


Advantage:
- per process - you need to do it for slow services only
- based on already implemented features


-----------
Conslusion:
-----------

I think the solutions 2.) is the only clear solution (address the basic
cause of the problem. Solution 4.) can be used as -i option alternative,
it is based on present features and is per-process.

Martin