Forum

Combined log Hiawatha+nginx

Aquanet
25 September 2014, 12:36
Hello Hugo,

We have one question about Hiawatha Monitor and Nginx logs.

Do you think we can reformat nginx logs to meet Hiawatha Monitor log format and then let Monitor import those logs too.

As we are running Nginx in front of Hiawatha for some purpose and some logs get lots as Nginx caches content. Same applies to Varnish, as it has a better caching system than Hiawatha.

We would want to gather logs in one place.

Regards
Andrew
Hugo Leisink
25 September 2014, 13:30
It can of course be done. Take a look at src/monitor.c as it holds the information you seek. It's in the several snprintf() statements.

The other thing you need to know is that the Monitor collects this information via a script which connects to the webservers it monitors to download it directly. What you must do with nginx is create a virtual host with the hostname 'monitor', which should only be accessible to the Monitor server. The index of that website must be an XML file listing the logfiles. The best thing you can do is setup a Hiawatha webserver with monitoring enabled, connect to it and do a GET / HTTP/1.0 with the HTTP header Host set to 'monitor'. Write a CGI script for nginx that creates this output based in the logfiles which are written to the document root of that virtual host.
Aquanet
28 September 2014, 18:10
Thank you, Hugo,

And how does the XML look like, I cant find it where it is currently stored on the hiawatha web servers, to see an example.

Regards
Andrew
Hugo Leisink
28 September 2014, 20:56
An example XML output:
<?xml version="1.0" encoding="ISO-8859-1"?>
<index>
<hostname>monitor</hostname>
<request_uri>/</request_uri>
<files>
<file type="file" timestamp="28 Sep 2014, 20:50:51" size="167 byte" extension="gz" url_encoded="1411930251.txt.gz">1411930251.txt.gz</file>
<file type="file" timestamp="28 Sep 2014, 20:52:08" size="207 byte" extension="gz" url_encoded="1411930328.txt.gz">1411930328.txt.gz</file>
<file type="file" timestamp="28 Sep 2014, 20:53:25" size="169 byte" extension="gz" url_encoded="1411930405.txt.gz">1411930405.txt.gz</file>
<file type="file" timestamp="28 Sep 2014, 20:54:42" size="230 byte" extension="gz" url_encoded="1411930482.txt.gz">1411930482.txt.gz</file>
</files>
<total_size>773 byte</total_size>
<software>Hiawatha v9.8</software>
</index>

It is the same as generated for the directory index. Use ShowIndex = xml to generate your own XML.
This topic has been closed.