[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Gluster-devel] [RFC] Enable message IDs in Gluster logs
From: |
Shyamsundar Ranganathan |
Subject: |
[Gluster-devel] [RFC] Enable message IDs in Gluster logs |
Date: |
Tue, 3 Dec 2013 07:09:59 -0500 (EST) |
Problem statement: Currently there are quite a slew of logs in Gluster that do
not lend themselves to trivial analysis by various tools that help collect and
monitor logs.
This FEAT is to make this _tooling_ better by giving logs some form of message
IDs so that the tools do not have to do complex log parsing to break it down to
problem areas and suggest troubleshooting options.
It is also intended in this RFE, to generate a catalog of such IDs and
messages, that can then be elaborated with various troubleshooting and
resolution options.
The idea behind the catalog is that, at each release point we have a catalog
for that release (updated during further maintenance releases), which can help
point to some documentation on common errors and how to recover from them. So
the message IDs do not have to be,
- unique across releases
- possibly needs little reconciliation on an ongoing basis with the catalog,
based on string changes
- does not have to be unique for each message, as the documentation can cover
collisions to provide clarity
Proposed solution: Add a hash of the message format to the log message, to
serve as the message ID.
IOW, the FMT in gf_log(dom, levl, fmt...) (and equivalents) is used to generate
a 32bit hash and the log is printed with the hash. As an example,
I [socket.c:3533:socket_init][29004409] 0-testv-client-2: SSL support is NOT
enabled
Advantages:
- Minimal code change (only the gf_log_* needs to change to further compute a
hash of the message and add it to the log)
- No developers need to remember to generate a message ID before adding any
messages etc.
- Cost of hashing during logging as computed using an example proves the
addition to be negligible (to almost no impact whatsoever for about 10K
messages printed)
Disadvantages:
- Hash collusion, hence 2 different messages having the same ID (but not an
issue due to the catalog/documentation as above)
- Hash changes due to changes in messages
- could be minor string edits lending to better messages, in which case hash
may change and we need to retain the same documentation (reconciliation of
catalog between releases)
- could be major string changes or parameter additions, in which case both
hash and the corresponding catalog documentation should change (hence not an
issue)
Catalogue generation would have to parse the code to extract all used format
strings (which is an interesting problem in itself).
This of this ala systemd/journald, without the message ID requirements from the
same.
Shyam
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Gluster-devel] [RFC] Enable message IDs in Gluster logs,
Shyamsundar Ranganathan <=