Weblog

21 May 2015, 13:21

Although the Hiawatha Monitor is available for more than 5 years (since Hiawatha v7.0), I think not many people actually use it. I'm not sure about the exact reason for that, but it might very well be that people don't understand the purpose or usefulness of this tool. Therefor, I'll explain in this weblog post what the Hiawatha Monitor can do for you.

While the main focus of the Hiawatha webserver is security, the Monitor's focus is more than that. Of course it includes security, but it's also about performance, availability and detecting errors. Via graphs with statistical information, you can have a quick view on what's going on at your webserver and which one needs attention. The purpose of the Monitor isn't to give you detailed information about what's going, but to simply give you enough pointers to start your investigation. You will know in what logfile you should start reading. For example, if the average amount of traffic for a certain website is usually between 300 and 400 megabytes and suddenly it generates 2 GB of traffic, you might want to check the access logfile for that website. The screenshot below shows that peak in traffic for a certain webserver.

By clicking on the bar, you get an overview of which website is responsible for what part of that amount of traffic, sorted by the amount. The one on top is most likely the one you should investigate.

In this example, the traffic peak was caused by my own web proxy, which I was developing at that moment. But it could also be someone downloading a large file from your website (should that file be there?) or your website was mentioned at another website (check Referer headers).

Amount of traffic is just one thing the Hiawatha Monitor shows you. It also shows you 404 errors (Not Found), which might indicate a dead link in one of your websites or 500 errors (Internal Server Error), which might indicate, for example, an error in one of your CGI scripts. CGI scripts might generate an error message or a warning, while the output was just fine. You won't see anything wrong at your website, while the error logfile grows larger and larger. The CGI errors graph at the CGI statistics section shows you that.

Hiawatha automatically reports failed logins for HTTP authentication, but failed logins for login functionality in your CGI scripts can also be monitored by printing the HTTP header X-Hiawatha-Monitor with the value 'failed_login'. The same goes for exploit attempts by using the value 'exploit_attempt'. Of course, don't forget to print detailed information about the exploit attempt to stderr for in your error logfile.

To make monitoring your Hiawatha webservers as easy as possible, the Monitor is also able to send you a daily report about the previous day. In one e-mail you get an easy overview about any significant changes in behavior and performance between yesterday and the last two weeks.

As you can see, the Hiawatha Monitor has a lot to offer in aiding you in maintaining your websites and webservers. Although I've told a lot about what I can do for you, there's much more to discover, like runtime of CGI scripts, checking if your webservers are up to date or messages from a CGI script directly to the Monitor. If you have multiple Hiawatha webservers running or just one, but with multiple websites, I'm very sure the Hiawatha Monitor will be a valuable tool.

Kapageridis Stavros
21 May 2015, 22:43
Hi Hugo, can you please help me with this error message ?
"PHP Notice: Use of undefined constant parent - assumed 'parent' in /home/hmonitor/libraries/graph_model.php on line 12".
Kapageridis Stavros
21 May 2015, 22:51
I am sorry forgot to mention that this message is returned when i run the cronjob
"59 23 * * * /path/to/monitor/website/database/send_reports".
Kapageridis Stavros
21 May 2015, 22:58
Another error message arise when i run the send_reports cronjob , the error msg is "PHP Notice: Undefined index: today in /home/hmonitor/database/send_reports on line 111"
Hugo Leisink
21 May 2015, 23:41
Put double quotes around the word parent to solve this issue.

I'll take a look at the undefined index message. Add a 'error_reporting(E_ALL & ~E_NOTICE);' at the start of the script as a work-around.
Heiko
22 May 2015, 00:11
I have no issue and I think it is very useful
me
22 May 2015, 09:28
The reason many don't use the monitor might be because they don't run production (or very important) sites with hiawatha. Otherwise I don't see a reason for them to not have it.

And as per your guess for some it could be lack of awareness (why don't you have the monitor's screen shot on the download page itself and a separate section on the download page for it just below the main download, moving the 'License' section to further bottom)

I really like it. When I start to run my site, I will really use the monitor (no need to install other log programs like webalizer and manually missing to view them time to time)
kapageridis Stavros
24 May 2015, 19:16
Hi Hugo, i follow your advice and there is only the error message for the undefinded index. After i add the error_reporting it goes. Thank you very much.
kapageridis Stavros
24 May 2015, 19:25
me :
As for the Monitor i don't think that it's usage is similar to other stats scripts. I like it cause it show me errors(cgi, not found) and the most important for me, exploit attempts, failed logins and events. Check your events in case you run a joomla or wordpress site and you will understand me.
borys
25 May 2015, 08:47
As for me, I use Hiawatha on my production VPS but don't use Hiawatha Monitor because of its PHP dependency. The reason why I don't (and won't) use PHP is off the topic. In the past I monitored my VPS using monit and collectd and I very much like these tools. I hope it would be nice to have more possibilities to monitor Hiawatha, not only one via utilizing PHP. Collectd uses plugins to monitor other processes. I believe these plugins are written in C. Hugo, what you think about writing a plugin for collectd which will make it possible to monitor Hiawatha?
Hugo Leisink
25 May 2015, 09:24
Sorry, won't do. The current Monitor tool (although written in PHP) works fine. I will cost me simply too much time to write a custom solution for everyone who doesn't like it. The reason why it's all open source is so everybody can make changes to it themselves. You want Hiawatha to send its log to collectd? Fine, go ahead!
borys
25 May 2015, 10:24
Oh, sorry... I just thought it would be very easy to write this plugin. At least it seemed to me, but I'm not C programmer.
emmel
26 May 2015, 07:29
Mainly, I'm not using it because of its dependencies. PHP is one thing, but having to use a MySQL database is something if a deal breaker for me. Now, the monitor is probably not as important for me as for some (non production, low traffic, private), but I guess I'm not alone with that.
Having to install (and run) another SQL database just for the Hiawatha Monitor is just too much. Now, if you supported different SQL database servers, or would just support sqlite that would make it much easier to use.
Hugo Leisink
26 May 2015, 09:00
The PHP framework that has been used for the Monitor has support for PostgreSQL. What you need is to add the PostgreSQL driver [github.com] and enable it in line 24 of public/index.php and in the scripts inside the database directory.

A lot of tools are available to convert MySQL databases to PostgreSQL, for example this one [github.com].

It shouldn't be too hard to make it work. I'll think about supporting it by default in future releases, but no promises yet.
emmel
26 May 2015, 11:11
Ah, thanks. I'll give postgres a shot later if I find the time.
emmel
26 May 2015, 13:10
Actually, turns out it's too much of a hassle to set up the tables.

The script you linked requires you to have the database running in MySQL, BTW, to do a more 'compatible' dump. Other scripts seem even harder to use, and I don't even want to think of the trouble differences in variable type will spell.

MySQL was never know to place nice with others. Compatibility between Oracle and PostgreSQL is a lot better, IIRC, which is rather ironic, considering.
Martin
19 June 2019, 19:46
It took some time to set up hiawatha monitor. Whatever I do, the statistics are empty. I only get "Warning, one or more webservers are unavailable or caused errors during syncing of the statistics!".
First problem is, I dont know if monitor is part of debian package. Compiled it extra with monitor. Still doesnt work.
If monitor and webserver are on same machine, I think IP 127.0.0.1 should work.
Its unclear, what to fill out in websites: only the servers were hiawatha is running or every virtual website?
"Hostname administration" is unclear too. If I push "Update hostnames" I get "» Error while updating hostname visibility."
Too many open questions for me ...