Drupal 6: Proposal for a watchdog hook for logging and alerts

Overview

Drupal has a module called watchdog that logs all messages to a database table. The module is currently mandatory, and hence writing to the table cannot be easily removed without hacking core. This can be a problem on sites with heavy traffic where the database is loaded with more logging unecessarily.

In issue #63881 from a while back on making watchdog a hook. The initial took a radical approach of making the watchdog module into its own watchdog.inc. This patch went no where, and Dries asked for a simple setting to enable either watchdog or syslog.

When I created such a patch, Dries revisited the idea of a hook, and this caused me to sit down and do some hacking.

The result of the hacking is a patch to core that:

  1. Makes watchdog optional, not mandatory.
  2. Retain all its features as they are present today.
  3. Preserve the watchdog() function API as it is today, insuring backward compatibility.
  4. Implement a new watchdog hook which modules can use to extend or replace the logging functionality of Drupal, and even push that into the realm of alerts, which we did not have in Drupal so far.

In addition to the core patch, I also wrote two modules, one that logs to the syslog, and another to send email alerts.

The patches and the modules are attached to the above issue.

Below are screen shots and examples of the various output.

Screen shots of the user interface

Under modules, there is now a new category called "Logging and alerts" that lists all the modules that implement the new watchdog hook. The site owner can choose none of the modules (no logging at all), or one or more modules (same message goes to different destinations).

Under "site configuration" there is a new entry for "Logging and alerts".

Clicking on it, all the modules that implement the hook are listed, with the site admin having the option to enable one or more of them, or even none of them.

Once a module is enable in the previous page, its settings will become visible under "site configuration", if it has any.

For example, the good old watchdog will show like this:

And the email log will show like this:

Clicking on each of the above, will show the settings page, as below.

Watchdog

This is no different than what is there today in 5.x and HEAD.

Email Logging and Alerts

The settings for the email log module is as follows.

As you can see each severity level can go to a different email address. So high priority stuff can be sent to a pager or mobile phone, while medium priority stuff can be send to a regular mailbox, and low priority can be sent to yet another mailbox.

When a log message/alert is issued, this is how the email looks like.

The message can be manipulated by a theme_...() function, as well as drupal_mail_alter().

To: sysadmin@example.com
Subject: [Drupal] Notice: Alert from your web site
From: Drupal <drupal@example.com>

Site: http://example.com
Severity: (0) Notice
Timestamp: 1175488384
Type: content
IP Address: 192.168.0.1
Request URI: http://example.com/?q=node/add/page
Referrer URI: http://example.com/?q=node/add/page
User: (1) admin
Link: view
Message:

page: added Yet another node.

Syslog

Syslog is the UNIX way of logging over a network. This is defined in RFC 3164. Windows supports syslog via third party software (free and commercial). The most simple form is to log messages to the file /var/log/syslog. However, there is an /etc/syslog.conf that defines where things go. In a large server farm, there would be a separate logging server that all the other servers send the messages too. If the organization has a centralized network management system like HP Open View, IBM Tivoli, or even Nagios, messages from the syslog can go to various destination depending on their origin and priority.

The syslog module maps the Drupal WATCHDOG severity levels to RFC 3164 values, and hands over the message to syslog. The message is completely themable.

Here is how a syslog message from Drupal looks like. The fields are separate by a | in case there is a need to parse it downstream in UNIX land.

Apr 2 00:28:08 server1 apache2: Drupal | http://example.com | 1175488087 | content | 1 | 192.168.0.249 | http://example.com/?q=node/add/page | http://example.com/?q=node/add/page | view | page: added Another node.

What about the future?

Well, this hook opens a lot of possibilities. One immediate benefit is that it avoids logging to the database on busy sites (e.g. Drupal.org), and offloads message handling to other means (e.g. syslog). The other benefit is opening the possibility of alerts, which was not possible before. Things like alerts to mobile phones, SMS, instant messaging (Jabber for example), SNMP or whatever you can imagine.

One can even write an NT log module for use of Windows system (this person will not be me!)

One thing I like to add after this patch is accepted is to expand the number of severities from the current three (ERROR, WARNING, NOTICE) to at least five (CRITICAL, ERROR, WARNING, NOTICE, INFO). Critical can then be the only one sent to pagers and the like, and the rest handled only by syslog for example.

The above mentioned RFC defines a lot of levels, but they are too many in my opinion.

0 Emergency: system is unusable
1 Alert: action must be taken immediately
2 Critical: critical conditions
3 Error: error conditions
4 Warning: warning conditions
5 Notice: normal but significant condition
6 Informational: informational messages
7 Debug: debug-level messages

So, go ahead and review the patch, and vote for it in issue #63881.

Contents: 

Comments

Good write up

I've pointed various people at the patch for this. We should definitely have this in Drupal 6. We'll do some testing and comment in the issue tracker follow ups. Some performance testing would be interesting.

Khalid, This is an excellent

Khalid,

This is an excellent proposal. Its clear that you've thought this out well, and I want to thank you for spending your time on this.

Committed to 6.x

After some discussions, and revised patches, Dries committed a modified version.The screens do not look exactly like the screen shots above, here are the highlights:

  • The settings for all the logging/alerts modules are all now under admin/settings/logging. Each module has its own page under that section.
  • The watchdog module has been renamed to dblog, with more or less the same functionality.
  • The syslog module is now in core.
  • I started a contributed logging and alerts project, and included the emaillog module in it, which is usable with 6.x as of today.

In other news, there is a bounty on porting logging functionality to Drupal 5.x, thanks to Boris. -- Khalid Baheyeldin

Hi, Thanks for very

Hi, Thanks for very interesting article. I really enjoyed reading all of your articles.

We have to wait for Drupal 6??

This is totally useful.. I've just spent a bit of time looking at this new framework as it would apply to Drupal 5..
http://drupal.org/node/149341

Doesn't seem to be quite there yet, some of the functionality seems to be.

Mike