Forum

I'm having a sever issue getting Ruby on Rails (http://rubyonrails.org) to run with Hiawatha. I've managed to get a bit further than Ricard Forniol Agust (this thread [www.hiawatha-webserver.org]) with my home page showing properly.

What I cannot find a way to do is: tell Hiawatha to forward any request of the type http: //domain.tld/controller or http://domain.tld/controller/action to the fcgi process to handle it. Example: http: //domain.tld/login

Basically, this is because there is no file extension in this case and the 'Extension' directive of the 'FastCGIserver' section does not recognize an asterisk as a significant parameter:

# Does NOT work  
FastCGIserver {
        FastCGIid = redmine
        ConnectTo = 127.0.0.1:2010
        Extension = fcgi, *
}

Spent quite a lot of time to no avail. Any help would be greatly appreciated!
yassen

Have you tried it without the Extension setting?

FastCGIserver {
        FastCGIid = redmine
        ConnectTo = 127.0.0.1:2010
}

Hugo, thanks; I surely did but still hiawatha was trying to find a file on the local filesystem and gave back 404.

I finally gave up and (while almost randomly changing the configuration for the last time), it worked out! I am playing with the configuration now (as I still don't know how I did it!) and will post back all relevant info in a while. I would appreciate your feedback in order to really understand what happens, and maybe we can compose a wiki page as I guess there will be other users interested.

Thanks again,
yassen

Here's what I have finally. This is for the redmine application (http://redmine.org) but may work for other rails apps too. I tried to avoid redmine-specific settings.

#httpd.conf:

ConnectionsTotal = 150
ConnectionsPerIP = 10
SystemLogfile = /var/log/hiawatha/system.log
GarbageLogfile = /var/log/hiawatha/garbage.log

Binding {
        Port = 80
}

Hostname = 127.0.0.1
WebsiteRoot = /var/www/hiawatha
StartFile = index.html
AccessLogfile = /var/www/hiawatha/access.log
ErrorLogfile = /var/www/hiawatha/error.log

## This is the interesting part:

FastCGIserver {
        FastCGIid = redmineFcgi
        ConnectTo = 127.0.0.1:2010
        Extension = fcgi
}

UrlToolkit {
        ToolkitID = redmineTk
        RequestURI exists Return
        Match ^(.*)$ Rewrite /dispatch.fcgi
}

VirtualHost {
        Hostname = redmine.lan
        WebsiteRoot = /root/works/redmine/public
        StartFile = dispatch.fcgi
        AccessLogfile = /var/www/default/access.log
        ErrorLogfile = /var/www/default/error.log
        FastCGI = redmineFcgi
        UseToolkit = redmineTk
}

# redmine/conf/environment.rb:

ENV['RAILS_ENV'] ||= 'production'    ## <<--- the only thing I've added is this line

RAILS_GEM_VERSION = '2.1.1' unless defined? RAILS_GEM_VERSION

require File.join(File.dirname(__FILE__), 'boot')

begin
  require File.join(File.dirname(__FILE__), '../vendor/plugins/engines/boot')
rescue LoadError
end

Rails::Initializer.run do |config|
  config.load_paths += %W( #{RAILS_ROOT}/app/sweepers )
  config.action_controller.session_store = Store
  config.active_record.observers = :message_observer
  config.action_mailer.perform_deliveries = false
end

Started the fcgi listeners via

cgi-fcgi -start -connect :2010 /root/works/redmine/public/dispatch.fcgi 3

(It is obvious that the root of the redmine install is /root/works/redmine but let me mention it explicitly.)

Just noticed that the redmine wiki is NOT working, though. Trying to access the start page gives me 404, but I just discovered thatr and have no time to troubleshoot it right now. (Will come to that later.)

Any further assistance would be highly appreciated.

Please, try the following configuration:

FastCGIserver {
        FastCGIid = redmineFcgi
        ConnectTo = 127.0.0.1:2010
}

UrlToolkit {
        ToolkitID = redmineTk
        RequestURI exists Return
        Match .* FastCGI redmineFcgi
}

VirtualHost {
        Hostname = redmine.lan
        WebsiteRoot = /root/works/redmine/public
        AccessLogfile = /var/www/default/access.log
        ErrorLogfile = /var/www/default/error.log
        UseToolkit = redmineTk
}

Thank you very much for the suggestion. Just tried it but unfortunately, does not work. Hiawatha gives back 404 for any url I tried (e.g. http://redmine.lan/ , http://redmine.lan/login). The error status comes from hiawatha, while in the first case (above) the 404 for the wiki comes from the redmine code (which might be my problem configuring redmine, I have to check).

Here's the hiawatha access.log for these two:

10.0.0.41|Thu 27 Nov 2008 00:25:49 +0200|404|0||GET / HTTP/1.1|Host: redmine.lan|User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4|Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8|Accept-Language: en-us,en;q=0.5|Accept-Encoding: gzip,deflate|Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7|Keep-Alive: 300|Connection: keep-alive
10.0.0.41|Thu 27 Nov 2008 00:25:53 +0200|404|0||GET /login HTTP/1.1|Host: redmine.lan|User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4|Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8|Accept-Language: en-us,en;q=0.5|Accept-Encoding: gzip,deflate|Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7|Keep-Alive: 300|Connection: keep-alive|If-None-Match: "74176006405f83840bb884c13296b9d1"

One more note: confirmed that the wiki 404 problem I reported earlier has nothing to do with Hiawatha: the same response status is returned if I run it under the WEBrick server, so it is purely on the RoR side.

vmdebian:~/works/redmine# ./script/server -e production -p 80
=> Booting WEBrick...
=> Rails 2.1.1 application started on http://0.0.0.0:80
=> Ctrl-C to shutdown server; call with --help for options
[2008-11-27 00:50:39] INFO  WEBrick 1.3.1
[2008-11-27 00:50:39] INFO  ruby 1.8.5 (2006-08-25) [i486-linux]
[2008-11-27 00:50:39] INFO  WEBrick::HTTPServer#start: pid=2731 port=80
10.48.52.41 - - [27/Nov/2008:00:51:03 EET] "GET /wiki/bt-records HTTP/1.1" 404 3257
http://redmine.lan/projects/show/bt-records -> /wiki/bt-records

It seems there is a bug in Hiawatha. Can you change line 1013 in hiawatha.c from

if (session->uri_is_dir) {
    strcpy(session->file_on_disk + length, session->host->start_file);
} else {
    return 404;
}

to

if (session->uri_is_dir) {
    strcpy(session->file_on_disk + length, session->host->start_file);
} else if ((session->host->fast_cgi.size == 0) && (session->toolkit_fastcgi == NULL)) {
    return 404;
}

and in xslt.c line 200:

if (strcmp(session->extension, "xml") != 0) {
    return false;
}

to

if (session->extension == NULL) {
    return false;
} else if (strcmp(session->extension, "xml") != 0) {
    return false;
}

Please let me know of that solves your problem.

Thank you got the patches, Hugo! Unfortunately, the result is the same: 404 for both URLs from above:

10.0.0.41|Thu 27 Nov 2008 13:09:02 +0200|404|0||GET /login HTTP/1.1|Host: redmine.lan|User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4|Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8|Accept-Language: en-us,en;q=0.5|Accept-Encoding: gzip,deflate|Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7|Keep-Alive: 300|Connection: keep-alive|Cache-Control: max-age=0
10.0.0.41|Thu 27 Nov 2008 13:09:07 +0200|404|0||GET / HTTP/1.1|Host: redmine.lan|User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4|Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8|Accept-Language: en-us,en;q=0.5|Accept-Encoding: gzip,deflate|Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7|Keep-Alive: 300|Connection: keep-alive

Here's what I did to test it:

Downloaded the 6.10 source; patched the two mentioned files; the diff follows:

vmdebian:~/works/hiawatha/hiawatha-6.10# diff -u ~/backups/hiawatha.c.orig ./hiawatha.c
--- /root/backups/hiawatha.c.orig       2008-10-11 00:21:50.000000000 +0300
+++ ./hiawatha.c        2008-11-27 12:34:10.000000000 +0200
@@ -1010,7 +1010,7 @@
        if (*(session->file_on_disk + length - 1) == '/') {
                if (session->uri_is_dir) {
                        strcpy(session->file_on_disk + length, session->host->start_file);
-               } else {
+               } else if ((session->host->fast_cgi.size == 0) && (session->toolkit_fastcgi == NULL)) {
                        return 404;
                }
        } else if (session->uri_is_dir) {


vmdebian:~/works/hiawatha/hiawatha-6.10# diff -u ~/backups/xslt.c.orig ./xslt.c
--- /root/backups/xslt.c.orig   2008-10-01 01:13:57.000000000 +0300
+++ ./xslt.c    2008-11-27 12:34:16.000000000 +0200
@@ -197,7 +197,9 @@

        /* Check extension
         */
-       if (strcmp(session->extension, "xml") != 0) {
+       if (session->extension == NULL) {
+               return false;
+       } else if (strcmp(session->extension, "xml") != 0) {
                return false;
        }

Then:

$ ./configure
$ ./make deb
$ cd ..
$ apt-get remove hiawatha ## not sure if really necessary, but won't hurt
$ dpkg -i hiawatha_6.10_i386.deb

The config file is exactly the same that you have given; here it is for completeness:

ConnectionsTotal = 150
ConnectionsPerIP = 10
SystemLogfile = /var/log/hiawatha/system.log
GarbageLogfile = /var/log/hiawatha/garbage.log

Binding {
        Port = 80
}

Hostname = 127.0.0.1
WebsiteRoot = /var/www/hiawatha
StartFile = index.html
AccessLogfile = /var/www/hiawatha/access.log
ErrorLogfile = /var/www/hiawatha/error.log

## This is the interesting part:

FastCGIserver {
        FastCGIid = redmineFcgi
        ConnectTo = 127.0.0.1:2010
}

UrlToolkit {
        ToolkitID = redmineTk
        RequestURI exists Return
        Match .* FastCGI redmineFcgi
}

VirtualHost {
        Hostname = redmine.lan
        WebsiteRoot = /root/works/redmine/public
        AccessLogfile = /var/www/default/access.log
        ErrorLogfile = /var/www/default/error.log
        UseToolkit = redmineTk
}

Please let me know if you need me to test further any patches. Would be glad to help fixing this issue.
yassen

I've sent you an e-mail with a new version. Please try that one.

Please, change "RequestURI exists Return" to "RequestURI isfile Return"
Otherwise, when requesting "/", the FastCGI server is not used because "/" is the webroot directory, which exists....

!!WORKS!! !!WORKS!! !!WORKS!!

Hugo, thank you so much!

> Please, change "RequestURI exists Return" to "RequestURI isfile Return"

Of course, stupid me, I should have guessed that a lot earlier. Sorry for creating additional hassle.

I will be out of office until Monday noon, so I cannot post back the full solution. And maybe it will not be necessary to post it here but better create a wiki page (I can do that) and just refer to it here for people that may find the tread while searching for a solution of the same problem? And also, shall we publish the patches to code now or simply (some day) release a new version and urge rails lovers to upgrade?

Your feedback is highly appreciated. Your help and support is MUCH appreciated, too!
yassen

Excellent! No problem about the 'hassle'. It was educational for me too.

I'll release a new version soon, so a patch won't be necessary. I think this issue is perfect for the wiki. I think it would be nice to have a Hiawatha wiki page for every available framework/CMS.

Hugo, thank you for your reply!

I'll come back to this issue as soon as I can (most probably tomorrow, 2.12.). I plan to compose the wiki page and post a link to it here. I want to first cover installing rails + fcgi on Debian Etch (which is not exactly hiawatha-related, but is related to this issue ... suggestions from your side?) and then in a second part -- configuring hiawatha to serve rails apps via fcgi.

I may need login credentials for the wiki if possible... would you please create an account for me?

Thanks again, talk soon!
yassen

Finally back, couldn't make it earlier :-(

Now I have to revise my emotional feedback: there is one line that is not present in the httpd.conf you suggested, but without it the thing does not work. Here is the working config, and the mentioned line marked:

## This is the interesting part:

FastCGIserver {
        FastCGIid = redmineFcgi
        ConnectTo = 127.0.0.1:2011
}

UrlToolkit {
        ToolkitID = redmineTk
        RequestURI isfile Return
        Match .* FastCGI redmineFcgi
}

VirtualHost {
        Hostname = redmine.lan
        WebsiteRoot = /root/works/redmine/r2085/public
        AccessLogfile = /var/www/default/access.log
        ErrorLogfile = /var/www/default/error.log
        UseToolkit = redmineTk
        FastCGI = redmineFcgi    ## <-- removing this line causes 403 for / and 404 for /login URLs
}

My apologies for not noticing this during my last tests

Now why is this important for me? Because I need to run two (or more) instances of Redmine, each one using a different database. Here is the configuration that I hope to get running (but cannot for now):

## This is the interesting part:

FastCGIserver {
        FastCGIid = oneRedmineFcgi
        ConnectTo = 127.0.0.1:2010
}


FastCGIserver {
        FastCGIid = twoRedmineFcgi
        ConnectTo = 127.0.0.1:2011
}


UrlToolkit {
        ToolkitID = oneTk
        Match ^/one(.*)$ Rewrite $1 Continue
        Match .* FastCGI oneRedmineFcgi
}


UrlToolkit {
        ToolkitID = twoTk
        Match ^/two(.*)$ Rewrite $1 Continue
        Match .* FastCGI twoRedmineFcgi
}


UrlToolkit {
        ToolkitID = redmineTk
        RequestURI isfile Return
        Match ^/one(.*)$  Goto  oneTk
        Match ^/two(.*)$  Goto  twoTk
        Match .*          DenyAccess
}


VirtualHost {
        Hostname = redmine.lan
        WebsiteRoot = /root/works/redmine/r2085/public
        AccessLogfile = /var/www/default/access.log
        ErrorLogfile = /var/www/default/error.log
        UseToolkit = redmineTk
        FastCGI = oneRedmineFcgi, twoRedmineFcgi   ## <-- hiawatha returns 403 and 404 if this line is removed
                                                   ##     if not removed/commented, the redmine code returns 404
}

I am pretty much stuck with this, Any help will be much appreciated, thanks in advance!
yassen

Addition to the above post:

In case the line at stake is left active in the config, the proper redmine instance is invoked, but with the original URL (no rewriting seems to have taken place). Thus Redmine returning 404 is a proper behavior from its own point of view.

For example, if I request http://redmine.lan/one/login, the redmine code gets /one/login and correctly returns 404 as it is not able to deal with the base part. I am not sure how much of this is actually a hiawatha problem, but in any case, the rewriting defined in oneTk/twoTk seems to not happen for some reason.

Hi Yassen, I'll try to look into this issue this evening.

Thank you so much, Hugo!

Here I have a slightly corrected httpd.conf, but alas, the rewriting in oneTk and twoTk still does not take place; hiawatha forwards the request to the correct redmine instance, yet without rewriting the URL (I need the /one and /two segments removed, to let Redmine take care for the rest, otherwise it says 404).

## This is the interesting part:

FastCGIserver {
        FastCGIid = oneRedmineFcgi
        ConnectTo = 127.0.0.1:2001
}

FastCGIserver {
        FastCGIid = twoRedmineFcgi
        ConnectTo = 127.0.0.1:2002
}

UrlToolkit {
        ToolkitID = oneTk
        Match ^/one(.*)$ Rewrite $1 Continue
        RequestURI isfile Return
        Match .* FastCGI oneRedmineFcgi
}

UrlToolkit {
        ToolkitID = twoTk
        Match ^/two(.*)$ Rewrite $1 Continue
        RequestURI isfile Return
        Match .* FastCGI twoRedmineFcgi
}

UrlToolkit {
        ToolkitID = redmineTk
        RequestURI isfile Return
        Match ^/one(.*)$  Goto  oneTk
        Match ^/two(.*)$  Goto  twoTk
        Match .*          DenyAccess
}

Thanks again for your tremendous help!
yassen

Sorry, this fragment slept out during copy-paste, last part of the above config:

VirtualHost {
        Hostname = redmine.lan
        WebsiteRoot = /root/works/redmine/r2085/public
        AccessLogfile = /var/www/default/access.log
        ErrorLogfile = /var/www/default/error.log
        UseToolkit = redmineTk
        FastCGI = oneRedmineFcgi, twoRedmineFcgi   ## <-- hiawatha returns 403 and 404 if this line is removed
                                                   ##     if not removed/commented, the redmine code returns 404
}

I've tried to use 2 FastCGI servers in one virtual host and it worked perfectly for me. What I have done looks like this:

FastCGIserver {
        FastCGIid = oneRedmineFcgi
        ConnectTo = 127.0.0.1:2001
}

FastCGIserver {
        FastCGIid = twoRedmineFcgi
        ConnectTo = 127.0.0.1:2002
}

UrlToolkit {
        ToolkitID = remove_one
        Match /one/(.*) Rewrite /$1 Continue
        Match .* FastCGI oneRedmineFcgi
}
UrlToolkit {
        ToolkitID = remove_two
        Match /two/(.*) Rewrite /$1 Continue
        Match .* FastCGI twoRedmineFcgi
}
UrlToolkit {
        ToolkitID = redmineTk
        RequestURI isfile Return
        Match ^/one/  Goto remove_one
        Match ^/two/  Goto remove_two
        Match .*          DenyAccess
}

I didn't use a FastCGI setting in the VirtualHost block.

If this is not working for you, I think it's very likely it's a Redmine issue.

Hugo, thank you so much for your concern! I'll probably disappoint you (forgive me) but I reproduce your config verbatim (find it below) and now hiawatha gives me back 403 without forwarding anything to the fcgi daemons

I'll paste the full config as I already don't know where to look for the problem ... I'm lost, really. I need this to run, I really do. Please help me if you are willing to. (Ahh, and the version I am running is the 6.11 you've sent me; do you use the same code for your setup?)

ConnectionsTotal = 150
ConnectionsPerIP = 10
SystemLogfile = /var/log/hiawatha/system.log
GarbageLogfile = /var/log/hiawatha/garbage.log

Binding {
        Port = 80
}

Hostname = 127.0.0.1
WebsiteRoot = /var/www/hiawatha
StartFile = index.html
AccessLogfile = /var/www/hiawatha/access.log
ErrorLogfile = /var/www/hiawatha/error.log

## This is the interesting part:

FastCGIserver {
        FastCGIid = oneRedmineFcgi
        ConnectTo = 127.0.0.1:2001
}

FastCGIserver {
        FastCGIid = twoRedmineFcgi
        ConnectTo = 127.0.0.1:2002
}

UrlToolkit {
        ToolkitID = remove_one
        Match /one/(.*) Rewrite /$1 Continue
        Match .* FastCGI oneRedmineFcgi
}

UrlToolkit {
        ToolkitID = remove_two
        Match /two/(.*) Rewrite /$1 Continue
        Match .* FastCGI twoRedmineFcgi
}

UrlToolkit {
        ToolkitID = redmineTk
        RequestURI isfile Return
        Match ^/one/  Goto remove_one
        Match ^/two/  Goto remove_two
        Match .*          DenyAccess
}

VirtualHost {
        Hostname = redmine.lan
        WebsiteRoot = /root/works/redmine/r2085/public
        AccessLogfile = /var/www/default/access.log
        ErrorLogfile = /var/www/default/error.log
        UseToolkit = redmineTk
}

Just for completeness, here are the hiawatha logs of the problematic requests (the fcgi deamons' logs are empty during these requests):

10.0.0.41|Fri 05 Dec 2008 01:39:31 +0200|404|0||GET /two/login HTTP/1.1|Host: redmine.lan|User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4|Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8|Accept-Language: en-us,en;q=0.7,bg;q=0.3|Accept-Encoding: gzip,deflate|Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7|Keep-Alive: 300|Connection: keep-alive
10.0.0.41|Fri 05 Dec 2008 01:40:54 +0200|403|0||GET /two/login HTTP/1.1|Host: redmine.lan|User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4|Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8|Accept-Language: en-us,en;q=0.7,bg;q=0.3|Accept-Encoding: gzip,deflate|Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7|Keep-Alive: 300|Connection: keep-alive
10.0.0.41|Fri 05 Dec 2008 01:45:01 +0200|403|0||GET /one/ HTTP/1.1|Host: redmine.lan|User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4|Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8|Accept-Language: en-us,en;q=0.7,bg;q=0.3|Accept-Encoding: gzip,deflate|Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7|Keep-Alive: 300|Connection: keep-alive
10.0.0.41|Fri 05 Dec 2008 01:45:07 +0200|403|0||GET /one/login HTTP/1.1|Host: redmine.lan|User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4|Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8|Accept-Language: en-us,en;q=0.7,bg;q=0.3|Accept-Encoding: gzip,deflate|Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7|Keep-Alive: 300|Connection: keep-alive

(The first line is not relevant, please disregard it; it has been generated while using a different httpd.conf)

Cleaning up a little here ... to make the (VERY) long story short, I finally did it after long weeks of struggle.

I modified the Rails routes.rb generally following the instructions of an article found here: http://wiki.rubyonrails.com/rails/pages/HowToInstallApplicationsInSubdirectories (originally posted here [www.riscosopen.org] )

Just a warning to people who may try to follow my footsteps: I had to change things compared to what is in the guide mentioned. Blindly following it literally would NOT work for me and will NOT work for you either! (Questions? Well, read ahead.)

I've posted a HowTo into this Hiawatha Wiki page [wiki.hiawatha-webserver.org] about the easier part of the work -- getting a usual Rails (Redmine) installation to work with a single database. As soon as I get enough time, I'll post the (shorter but more interesting) part about getting Rails to serve multiple Redmine instances with a single install.

Cheers,
yassen