Drupal 6: Proposal for a watchdog hook for logging and alerts

Overview

Drupal has a module called watchdog that logs all messages to a database table. The module is currently mandatory, and hence writing to the table cannot be easily removed without hacking core. This can be a problem on sites with heavy traffic where the database is loaded with more logging unecessarily.

In issue #63881 from a while back on making watchdog a hook. The initial took a radical approach of making the watchdog module into its own watchdog.inc. This patch went no where, and Dries asked for a simple setting to enable either watchdog or syslog.

When I created such a patch, Dries revisited the idea of a hook, and this caused me to sit down and do some hacking.

The result of the hacking is a patch to core that:

1. Makes watchdog optional, not mandatory.
2. Retain all its features as they are present today.
3. Preserve the watchdog() function API as it is today, insuring backward compatibility.
4. Implement a new watchdog hook which modules can use to extend or replace the logging functionality of Drupal, and even push that into the realm of alerts, which we did not have in Drupal so far.

In addition to the core patch, I also wrote two modules, one that logs to the syslog, and another to send email alerts.

The patches and the modules are attached to the above issue.

Below are screen shots and examples of the various output.

Screen shots of the user interface

Under modules, there is now a new category called "Logging and alerts" that lists all the modules that implement the new watchdog hook. The site owner can choose none of the modules (no logging at all), or one or more modules (same message goes to different destinations).

Under "site configuration" there is a new entry for "Logging and alerts".

Clicking on it, all the modules that implement the hook are listed, with the site admin having the option to enable one or more of them, or even none of them.

Once a module is enable in the previous page, its settings will become visible under "site configuration", if it has any.

For example, the good old watchdog will show like this:

And the email log will show like this:

Clicking on each of the above, will show the settings page, as below.

Watchdog

This is no different than what is there today in 5.x and HEAD.

Email Logging and Alerts

The settings for the email log module is as follows.

As you can see each severity level can go to a different email address. So high priority stuff can be sent to a pager or mobile phone, while medium priority stuff can be send to a regular mailbox, and low priority can be sent to yet another mailbox.

When a log message/alert is issued, this is how the email looks like.

The message can be manipulated by a theme_...() function, as well as drupal_mail_alter().

To: sysadmin@example.com
Subject: [Drupal] Notice: Alert from your web site
From: Drupal <drupal@example.com>

Site:         http://example.com
Severity:     (0) Notice
Timestamp:    1175488384
Type:         content
IP Address:   192.168.0.1
Request URI:  http://example.com/?q=node/add/page
Referrer URI: http://example.com/?q=node/add/page
User:         (1) admin
Link:         view
Message:

page: added Yet another node.

Syslog

Syslog is the UNIX way of logging over a network. This is defined in RFC 3164. Windows supports syslog via third party software (free and commercial). The most simple form is to log messages to the file /var/log/syslog. However, there is an /etc/syslog.conf that defines where things go. In a large server farm, there would be a separate logging server that all the other servers send the messages too. If the organization has a centralized network management system like HP Open View, IBM Tivoli, or even Nagios, messages from the syslog can go to various destination depending on their origin and priority.

The syslog module maps the Drupal WATCHDOG severity levels to RFC 3164 values, and hands over the message to syslog. The message is completely themable.

Here is how a syslog message from Drupal looks like. The fields are separate by a | in case there is a need to parse it downstream in UNIX land.

Apr 2 00:28:08 server1 apache2: Drupal | http://example.com | 1175488087 | content | 1 | 192.168.0.249 | http://example.com/?q=node/add/page | http://example.com/?q=node/add/page | view | page: added Another node.

What about the future?

Well, this hook opens a lot of possibilities. One immediate benefit is that it avoids logging to the database on busy sites (e.g. Drupal.org), and offloads message handling to other means (e.g. syslog). The other benefit is opening the possibility of alerts, which was not possible before. Things like alerts to mobile phones, SMS, instant messaging (Jabber for example), SNMP or whatever you can imagine.

One can even write an NT log module for use of Windows system (this person will not be me!)

One thing I like to add after this patch is accepted is to expand the number of severities from the current three (ERROR, WARNING, NOTICE) to at least five (CRITICAL, ERROR, WARNING, NOTICE, INFO). Critical can then be the only one sent to pagers and the like, and the rest handled only by syslog for example.

The above mentioned RFC defines a lot of levels, but they are too many in my opinion.

0 Emergency: system is unusable
1 Alert: action must be taken immediately
2 Critical: critical conditions
3 Error: error conditions
4 Warning: warning conditions
5 Notice: normal but significant condition
6 Informational: informational messages
7 Debug: debug-level messages

So, go ahead and review the patch, and vote for it in issue #63881.