Forum

Hiawatha version: 9.3.1 ( installed from Debian amd64 debian package )
Operating System: Debian GNU/Linux 7 (wheezy)

A sudden warning/error condition occured.when running the below command. The indication for this was all virtual hosts returned 403, there are NO 403's logged. Only change prior to this situation was adding ReverseProxy and AccessList configuration options to an unrelated virtual host. Even with fresh hiawatha.conf this keeps happening.

service hiawatha start
Stopping webserver: Hiawatha
Starting webserver: Hiawatha
Warning: error redirecting stdin
Warning: error redirecting stdout

Logs show there is an issue with php5-fpm, the configuration did not change.

Sun 05 Jan 2014 12:51:03 +0100|FastCGI server PHP5 is still (partially) unavailable
11.22.33.44|Sun 05 Jan 2014 12:51:41 +0100|Timeout while waiting for request
11.22.33.44|Sun 05 Jan 2014 12:51:41 +0100|Timeout while waiting for request
11.22.33.44|Sun 05 Jan 2014 12:51:41 +0100|Timeout while waiting for request
11.22.33.44|Sun 05 Jan 2014 12:51:41 +0100|Timeout while waiting for request
11.22.33.44|Sun 05 Jan 2014 12:51:42 +0100|Timeout while waiting for request
Sun 05 Jan 2014 12:52:40 +0100|can't connect to FastCGI server PHP5
Sun 05 Jan 2014 12:53:12 +0100|FastCGI server PHP5 is still (partially) unavailable
Sun 05 Jan 2014 12:53:42 +0100|Hiawatha v9.3.1 stopped
Sun 05 Jan 2014 12:53:43 +0100|Hiawatha v9.3.1 started

I am not keen on changing any permissions on /dev/null as suggested earlier. It is my growing yet unvalidated impression Hiawatha is not very reliable for continued operation or i'm underestimating the complexity of it's set-up.

These kind of issues are always hard for me, because the cause lies almost everytime outside Hiawatha. I have no idea what goes on at your system, so I can't tell what causes this.

Tried restarting the PHP5 FastCGI daemon?

Thanks for the quick reply, PEBKAC eh ... well, i'm not afraid to fail anymore at this point ...

Google only returns a few results on "Warning: error redirecting stdin" "Warning: error redirecting stdout" , these are generated from hiawatha. The permission on /dev/null are unchanged and default as on other same OS sytems.

Restarted php-fpm , rebooted the system, same same.

http://fossies.org/linux/www/hiawatha-9.3.1.tar.gz:a/hiawatha-9.3.1/src/hiawatha.c

900 /* Redirecting I/O to /dev/null
901 */
902 if (settings->daemon) {
903 if (close(STDIN_FILENO) == -1) {
904 fprintf(stderr, "Warning: error closing STDIN\n");
905 } else if (open("/dev/null", O_RDONLY) == -1) {
906 fprintf(stderr, "Warning: error redirecting stdin\n");
907 }
908 if (close(STDOUT_FILENO) == -1) {
909 fprintf(stderr, "Warning: error closing STDOUT\n");
910 } else if (open("/dev/null", O_WRONLY) == -1) {
911 fprintf(stderr, "Warning: error redirecting stdout\n");
912 }
913 if (close(STDERR_FILENO) == -1) {
914 fprintf(stderr, "Warning: error closing STDERR\n");
915 } else if (open("/dev/null", O_WRONLY) == -1) {
916 log_string(config->system_logfile, "Warning: error redirecting stderr\n");
917 }
918 }

I've trimmed down hiawatha.conf to only serve a single virtualhost serving static html, not load any CGI, not load any FastCGI. Same problem, same warning message "Warning: error redirecting .... " These message ( understandably ) occur on both start and force-reload commands for the service. Also when just starting hiawatha from the CLI but not when starting like `hiawatha -d` Though the pages keep returning a 403.

I'm in doubt if i understand you're reasoning the cause lies outside of Hiawatha. Though i would not be stumped if the cause did situate outside of Hiawatha. I hope to find the fix for this issue and document it here.

You're right, the stdin/stdout messages are from Hiawatha. I was only referring to the PHP FastCGI daemon warnings.

Could it be that you are using SELinux, GRSecurity or AppArmor?

Just checked

apparmor_status

AppArmor available in kernel.
AppArmor not enabled. (Kernel not booted with "security=apparmor"?)

Anyway, if Hiawatha says it can't redirect stdin and stdout, it has something to do with access rights. I assumed you are starting and stopping Hiawatha and PHP as root? Is that correct?

HIawatha is running with ServerId = web:web and so is the php-fpm daemon. These are obviously started as root but change privileges.

The /var/www/website tree has been given extreme privileges using chmod -R 777 /var/www/website and so was /var/www and at one point even /var

I've also cleared browser cache, restarted the browser and used a different browser. There is no doubt the error is generated by hiawatha, at this point i'm not sure if receives reply from my server or another one somewhere in the world. As i see no messages in the logs anymore. I'm afraid this might need to be a system reinstall.

I'm looking to recompile hiawatha with debug flags set.

I've recompiled with the debug information and tomahawk installed. This is the first time i've ran into such an issue with Hiawatha. Any pointers on debugging and troubleshooting are welcome.

I found a way to simulate the warning/error, which explains why initially it seemed to originate from hiawatha only.

setsid /usr/sbin/hiawatha

Extracted and validated based on the below source code (i hope this is visible because the paste appears as a listof numbers only

if (settings->daemon) {
663 switch (pid = fork()) {
664 case -1:
665 perror("fork()");
666 return -1;
667 case 0:
668 if (setsid() == -1) {
669 perror("setsid()");
670 return -1;
671 }
672 break;
673 default:
674 log_pid(config, pid, config->server_uid);
675 return 0;
676 }
677 } else {
678 log_pid(config, getpid(), config->server_uid);
679 }
680

The thing is: I've never had any complains about stdin/stdout and setsid. It worked fine for more than 10 years. You are the first one having problems with it. I'm very sure it has something to do with your system, not with Hiawatha. As I have no access to your system (and I don't want it ), I can't tell what goes wrong.