Forum

I have the problem where if Hiawatha is currently running and I try to restart, it does not restart. I have to stop the service and then start the service in order for it to work again. It also works if I run another restart command.

Here are the entries from syslog with notes on what commands were issued. This is from a fresh install on Debian 8.7 with Hiawatha 10.5. I also verified the PID file was removed correctly after restarts and stops were issued.

==> Clean startup:  service hiawatha start

Jan 31 18:29:47 digitalocean systemd[1]: Starting LSB: Hiawatha webserver...
Jan 31 18:29:47 digitalocean hiawatha[2160]: Starting Hiawatha webserver: hiawatha.
Jan 31 18:29:47 digitalocean systemd[1]: Started LSB: Hiawatha webserver.

===> Restart issued (fails):  service hiawatha restart

Jan 31 18:30:16 digitalocean systemd[1]: Stopping LSB: Hiawatha webserver...
Jan 31 18:30:16 digitalocean hiawatha[2198]: Stopping Hiawatha webserver: hiawatha.
Jan 31 18:30:16 digitalocean systemd[1]: Starting LSB: Hiawatha webserver...
Jan 31 18:30:16 digitalocean hiawatha[2206]: Starting Hiawatha webserver: hiawathaError binding 0.0.0.0:80
Jan 31 18:30:16 digitalocean hiawatha[2206]: failed!
Jan 31 18:30:16 digitalocean systemd[1]: Started LSB: Hiawatha webserver.

==> Second restart issued (works):  service hiawatha restart

Jan 31 18:30:46 digitalocean systemd[1]: Stopping LSB: Hiawatha webserver...
Jan 31 18:30:46 digitalocean hiawatha[2222]: hiawatha is not running ... failed!
Jan 31 18:30:46 digitalocean systemd[1]: hiawatha.service: control process exited, code=exited status=1
Jan 31 18:30:46 digitalocean systemd[1]: Unit hiawatha.service entered failed state.
Jan 31 18:30:46 digitalocean systemd[1]: Starting LSB: Hiawatha webserver...
Jan 31 18:30:46 digitalocean hiawatha[2230]: Starting Hiawatha webserver: hiawatha.
Jan 31 18:30:46 digitalocean systemd[1]: Started LSB: Hiawatha webserver.

==> Restart again (fails):  service hiwatha restart

Jan 31 18:34:00 digitalocean systemd[1]: Stopping LSB: Hiawatha webserver...
Jan 31 18:34:00 digitalocean hiawatha[2268]: Stopping Hiawatha webserver: hiawatha.
Jan 31 18:34:00 digitalocean systemd[1]: Starting LSB: Hiawatha webserver...
Jan 31 18:34:00 digitalocean hiawatha[2277]: Starting Hiawatha webserver: hiawathaError binding 0.0.0.0:80
Jan 31 18:34:00 digitalocean hiawatha[2277]: failed!
Jan 31 18:34:00 digitalocean systemd[1]: Started LSB: Hiawatha webserver.

==> Hiawatha is not running.  Issue start (fails):  service hiawatha start

Jan 31 18:34:54 digitalocean systemd[1]: Started LSB: Hiawatha webserver.

==> Hiawatha is still not running.  Start again (fails):  service hiawatha start

Jan 31 18:35:13 digitalocean systemd[1]: Started LSB: Hiawatha webserver.

==> Stop service (works):  service hiawatha stop

Jan 31 18:35:49 digitalocean systemd[1]: Stopping LSB: Hiawatha webserver...
Jan 31 18:35:49 digitalocean hiawatha[2341]: hiawatha is not running ... failed!
Jan 31 18:35:49 digitalocean systemd[1]: hiawatha.service: control process exited, code=exited status=1
Jan 31 18:35:49 digitalocean systemd[1]: Stopped LSB: Hiawatha webserver.
Jan 31 18:35:49 digitalocean systemd[1]: Unit hiawatha.service entered failed state.

==> Start service after stop command (works):  service hiawatha start

Jan 31 18:36:17 digitalocean systemd[1]: Starting LSB: Hiawatha webserver...
Jan 31 18:36:17 digitalocean hiawatha[2367]: Starting Hiawatha webserver: hiawatha.
Jan 31 18:36:17 digitalocean systemd[1]: Started LSB: Hiawatha webserver

What happens if you kill hiawatha manually (kill -15 <hiawatha pid>) and start hiawatha manually (run /usr/sbin/hiawatha)?

Ok, just ran the kill -15 command.

* Hiawatha process has terminated
* No log entries in syslog
* "Hiawatha v10.5 stopped." in hiawatha system.log file.
* PID file still exists at /var/run/hiawatha.pid

Now running /usr/sbin/hiawatha...

* Hiawatha process is running
* No log entries in syslog
* "Hiawatha v10.5 started." in hiawatha system.log file
* PID file replaced with correct PID

However if I stop hiawatha with "kill -15" then "service hiawatha start" does not work (tried multiple times). I still have to run "service hiawatha stop" first.

Another test. Just terminated the process again with "kill -15". This time running "service hiawatha restart" correctly starts hiawatha.

So, Hiawatha seems to work fine. The problem is in the start/stop script. Are you able to debug it?

Not really. From the logs above, my guess was that Hiawatha was not stopping quickly enough before it got restarted.

So, I did a test from the command line:

root@TestDroplet:~# export PIDFILE=/var/run/hiawatha.pid
root@TestDroplet:~# export DAEMON=/usr/sbin/hiawatha
root@TestDroplet:~# start-stop-daemon --stop --quiet --oknodo --retry 30 --pidfile $PIDFILE; start-stop-daemon --start --quiet --oknodo --pidfile $PIDFILE --exec $DAEMON
root@TestDroplet:~# ps -ef | grep hiawatha
www-data  1538     1  0 17:20 ?        00:00:00 /usr/sbin/hiawatha
root      1566   827  0 17:20 pts/0    00:00:00 grep hiawatha

Which seems to have worked just fine!

Wait a sec.... Looking through /var/log/syslog I notice that I never see a "Restarting" entry. Hmmm...

Line 57 of /etc/init.d/hiawatha, lets add the "--retry 30" switch. (Probably should have had that there anyway.)

                if start-stop-daemon --stop --quiet --oknodo --retry 30 --pidfile $PIDFILE; then

Ok, the restart works now! Running "service hiawatha restart" never actually runs the restart section of code. It must run stop and start separately instead. Looks like you can delete that section.

Thanks. I've removed the restart section and added the '--retry 30' for the next release.

Hugo, just a note that the restart section is probably still necessary if it is run as a SysVinit script.

Looks like when you include the /lib/lsb/init-functions it highjacks the request and runs it through systemctl instead.

Ok.