Weblog

26 July 2015, 11:57

In the past, I've playing with the idea of using an SQL database to store parts of the configuration, so one can add, change or remove website configurations without restarting the webserver. Recently, someone also asked about this in a forum post. This opion might also be usefull for people who own a lot of webservers.

There are of course many ways to implement this. A separate configuration for each webserver, multiple webservers using one and the same configuration, temporarily caching of configuration for improved speed, etc. What I'm looking for is people who have experience at maintaining large clusters of webservers who are willing to share their thoughts on this with me.

I started a poll to see how many people are actually interested in this feature. Hiawatha is not used by many people and this feature will require quite some work to implement. Of course, I won't go through all this trouble if not many people are interested in it.

metodi
27 July 2015, 14:50
you create this features hiawatha might become a threat to apache dominance!
Jeff
27 July 2015, 19:27
What about the ability to reload the configuration files without restarting the server before implementing this feature? In my case I wrote a custom web hosting panel and daemon to administer all the vhost, so in my humble opinion it would be far more important been able to reload the hiawatha configuration files without restarting. Also adding support for a client/server DBMS would add bloat that not everybody is going to need. In the case such option is needed SQLite should be taken into consideration since it requires far less runtime resources.
Alexander S
27 July 2015, 20:44
I ran about 30 webservers at peak times, it was painful.

A centralized MySQLbased config would be really awesome,
and another big reason for switching to hiawatha.
Hugo Leisink
27 July 2015, 21:01
@Jeff: Reloading configuration for a live webserver is far more complex than you probably realize. Previous Hiawatha versions allowed it, but it caused many stability issues. After the new configuration is loaded, you can't simple remove the old configuration from memory, because it's probably still in use. What if a browser already is connected and after reload opens new connections for the same website. Then some connections are handled via the old configuration and some via the new configuration.

@Alexander: Can you tell me what you really need in maintaining many webservers. What configuration changes are needed? Only adding / removing virtual hosts? Changes in virtual hosts? Adding support for MySQL is a big change and I really want to avoid it of not really need. Can you assist me in brainstorming about what is needed? What if for example I add a feature to only add new virtual hosts. And many remove virtual hosts. Will that work too?
DP Roberts
28 July 2015, 00:42
I like the idea however I have a few suggestions:

1) support both standard text or database configs
2) support for multiple database server types - sqlite, postgresql, mysql, etc... perhaps some nosql types
3) support remote database servers - for high availability environments
Paolo
28 July 2015, 00:54
In my personal experience, restarting the webserver is not a big issue.
Even if I manage a significant number of vhosts, nobody cares.

But I think that a database driven configuration would be a boost
to switch from Apache to Hiawatha because would be easier to
write down a configurator and try to fill the gap that
do not permit to have a wider base of "normal" users expecting
a "standard" LAMP server for their Wordpress, Joomla or so.
Only my 2 cents.
Andrew H
28 July 2015, 06:53
At my current work we use clusters of Apache servers using mod_vhost_alias [httpd.apache.org] to achieve something similar to this (I realise this uses directories on disk rather than configs in a database but it does mean we can dynamically add and remove vhosts without restarting/reloading).

Cheers,
Andrew
Hugo Leisink
28 July 2015, 09:20
@Andrew: Is restarting a webserver really such a big issue? Do new website really have to be online the minute you add them to the configuration? Is periodically checking for a configuration change and restarting the webserver if so an acceptable solution? This can, for example, be done once per hour or once per day. During a restart, the webserver will be offline for about 2 seconds. Is that really unacceptable?
John
28 July 2015, 09:32
@Hugo What is a pie without cherry? It's still good but it ain't finished. If you for example make it so rewrite rules can be loaded from a database as well it would be better if they are instantly loaded instead of the server being restarted once a hour, or once a day. For security reasons doing this through a web application ain't the smartest idea.
Andrew H
28 July 2015, 10:31
@Hugo restarting a webserver isn't the issue, it is done that way to automate the adding of vhosts through a control panel so it doesn't need any access to the servers (and is also a lot quicker than loading all the vhosts with apache - as a shared hosting provider we have thousands or sites!).

Admittedly this isn't an everyday problem for most people - on my own servers (where I run Hiawatha) I routinely retart the server if needed.

Cheers,
Andrew
Benjamin
28 July 2015, 21:35
NoSQL is probably easiest to integrate and work with since it's so flexible. But MySQL is the most used open source relational database.
Marius'95
8 August 2015, 19:39
Hugo,
There was a poll about Hiawatha usage. Hobby or personal website was 44%. That is the user's answer to your question. I didn't vote, but I'm one of those 44% and I don't need or want SQL configuration.

But this is your program, your creation. If you want to change it's main category of users, feel free to do it. It's what you think that matters in the end.

One other choice would be to fork it and mentain two branches: one for home users and one for corporate use.

Marius'95
Fred
11 November 2015, 11:50
Hi,

How do I vote? For me it'll be a life chamging feature
Fred
31 January 2016, 21:40
Hi Hugo,

Just wondering if you mafe a decision if you will implement this feature or not?
The way I go around it at the moment is that I have 1 webserver per domain behind hiawatha proxy...

I only need to restart the proxy one and if I made further change to the domain, i can restart again and agin whithout affecting other domain.
I achieve this using FreeBSD jail

Please post an update
Hugo Leisink
31 January 2016, 21:44
No, because there are administration tools available (like Salt and Puppet) that already offer this.