Apache instances/responsiveness issue...solved

mklein · Post by **mklein** » Sun Feb 22, 2009 7:29 pm

I cannot tell a lie...

Although lowering this setting helped me immeasurably (it did!) it's not "the fix" to apache issues.

I noticed night/day difference between 5 and 15 for keep-alive timeout...but something else can cause apache to be unresponsive (zm?) and it spawns more threads to handle...but none of them can it seems and up end up with slowly more workers.

So now and then I see this condition still...but it's vastly better than behaviour I had before.

phowarth · Post by **phowarth** » Mon Feb 23, 2009 12:30 am

Below is the output of top -u apache when the httpd is un-responsive. When the system has no/few connections the apache cpu reading is 2% to 3%.

The problem only occurs when I leave the montage running on my 3 camera system. (2 analog cams and 1 USB).

I have been using firefox and seamonkey but I guess I will have to break down and use explorer to see if the problem goes away.

My challenge with explorer is that the montage view shows the 3 camera views, but they all share the same image. (with jpeg) I will switch it over to see what the situation is with ffmeg streaming.

top - 19:23:22 up 1:28, 3 users, load average: 1.67, 1.25, 1.22
Tasks: 396 total, 2 running, 394 sleeping, 0 stopped, 0 zombie
Cpu(s): 41.7%us, 2.3%sy, 0.0%ni, 55.6%id, 0.0%wa, 0.3%hi, 0.0%si, 0.0%st
Mem: 2060248k total, 2002944k used, 57304k free, 40944k buffers
Swap: 3903752k total, 0k used, 3903752k free, 1414624k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3470 apache 20 0 201m 98m 90m R 17 4.9 14:52.63 zma
3723 apache 20 0 148m 45m 37m S 13 2.3 10:27.13 nph-zms
3718 apache 20 0 201m 98m 90m S 12 4.9 10:26.91 nph-zms
3474 apache 20 0 150m 46m 37m S 4 2.3 3:27.67 zmc
3462 apache 20 0 201m 98m 90m S 4 4.9 3:44.42 zma
3388 apache 20 0 60816 9184 4296 S 0 0.4 0:03.04 httpd
3389 apache 20 0 60816 9176 4288 S 0 0.4 0:02.99 httpd
3390 apache 20 0 60816 9432 4532 S 0 0.5 0:03.02 httpd
3391 apache 20 0 60868 9480 4528 S 0 0.5 0:00.20 httpd
3392 apache 20 0 61604 8040 2600 S 0 0.4 0:03.97 httpd
3396 apache 20 0 60816 9176 4288 S 0 0.4 0:03.93 httpd
3399 apache 20 0 61604 8004 2596 S 0 0.4 0:01.01 httpd
3400 apache 20 0 60816 9120 4232 S 0 0.4 0:01.35 httpd
3435 apache 20 0 11336 6140 1388 S 0 0.3 0:00.02 zmdc.pl
3458 apache 20 0 206m 103m 95m S 0 5.1 0:04.00 zmc
3466 apache 20 0 206m 103m 95m S 0 5.1 0:04.35 zmc
3478 apache 20 0 14480 10m 2768 S 0 0.5 0:00.27 zmfilter.pl

ammaross · Post by **ammaross** » Tue Mar 10, 2009 8:29 pm

I switched my httpd.conf (Fedora 9) to the KeepAliveTimeout of 5 and it "resolved" an issue I've been having with trying to pull up the ZM index page in FireFox taking up to 15-20s. Once in, the system seemed to work fine, it was just the initial connection that took forever. I'll have to add this to my server-setup info...

Thanks for the pointer.

I haven't ran into a "freezing" problem for ZM itself though. Perhaps that's because my system is on a Core2Quad with the MMX-enabled libJPEG...

phowarth · Post by **phowarth** » Fri Mar 13, 2009 12:16 am

I never did solve the problem of apache hanging with zoneminder.

In the end I just set up a cron job with the following

httpd -k restart

I run it at the top of every hour and have now been running ZM with Firefox for a few weeks without problem.

The only downside is the montage window needs to be manually refreshed after an apache restart.

Cheers.

kingofkya · Post by **kingofkya** » Sat Mar 14, 2009 10:58 pm

Wow this cut my lag time to server by about half. Nice Find

c0mputerking · Post by **c0mputerking** » Mon Mar 16, 2009 2:07 am

Hello all my firefox 3.0.0.7 is hanging it works ok for awhile example opens a montage window fine then a camera window then another camera window but if i close the camera window and try to open it again i just get a white screen that sits there forever. I have maxkeepalive set to 5 but it does not seem to be helping. Below is the output of my apachectl status command.

ps i have 4 cameras 640X480 (one set to grey to keep CPU lower) running through blue cherry card at 15 fps idle and 25 on alarm as well as one airlink747w at 640X480 system seems to be fairly stable other than i cannot connect ie zoneminder is not crashing or anything. Should will try lowering everything to 320X240 and less fps to see if that helps as memory always seems a bit on the short side.

em: 514824k total, 481124k used, 33700k free, 6332k buffers
Swap: 979956k total, 29816k used, 950140k free, 182136k cached

apachectl status
Apache Server Status for localhost

Server Version: Apache/2.2.11 (Unix) mod_ssl/2.2.11 OpenSSL/0.9.8j
DAV/2

Server Built: Jan 29 2009 15:13:41
__________________________________________________________________

Current Time: Sunday, 15-Mar-2009 19:56:47 MDT
Restart Time: Sunday, 15-Mar-2009 18:48:40 MDT
Parent Server Generation: 0
Server uptime: 1 hour 8 minutes 6 seconds
Total accesses: 56 - Total Traffic: 253 kB
CPU Usage: u1.31 s.19 cu.74 cs0 - .0548% CPU load
.0137 requests/sec - 63 B/second - 4626 B/request
81 requests currently being processed, 6 idle workers

WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
WWWWWWWWWWWWWW_W_WW____.........................................
................................................................
................................................................

Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R" Reading Request,
"W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully finishing,
"I" Idle cleanup of worker, "." Open slot with no current process
[root@1 user]#

c0mputerking · Post by **c0mputerking** » Mon Mar 16, 2009 2:17 am

Hello all my firefox 3.0.0.7 is hanging it works ok for awhile example opens a montage window fine then a camera window then another camera window but if i close the camera window and try to open it again i just get a white screen that sits there forever. I have maxkeepalive set to 5 but it does not seem to be helping. Below is the output of my apachectl status command.

ps i have 4 cameras 640X480 (one set to grey to keep CPU lower) running through blue cherry card at 15 fps idle and 25 on alarm as well as one airlink747w at 640X480 system seems to be fairly stable other than i cannot connect ie zoneminder is not crashing or anything. Should will try lowering everything to 320X240 and less fps to see if that helps as memory always seems a bit on the short side.

em: 514824k total, 481124k used, 33700k free, 6332k buffers
Swap: 979956k total, 29816k used, 950140k free, 182136k cached

apachectl status
Apache Server Status for localhost

Server Version: Apache/2.2.11 (Unix) mod_ssl/2.2.11 OpenSSL/0.9.8j
DAV/2

Server Built: Jan 29 2009 15:13:41
__________________________________________________________________

Current Time: Sunday, 15-Mar-2009 19:56:47 MDT
Restart Time: Sunday, 15-Mar-2009 18:48:40 MDT
Parent Server Generation: 0
Server uptime: 1 hour 8 minutes 6 seconds
Total accesses: 56 - Total Traffic: 253 kB
CPU Usage: u1.31 s.19 cu.74 cs0 - .0548% CPU load
.0137 requests/sec - 63 B/second - 4626 B/request
81 requests currently being processed, 6 idle workers

WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
WWWWWWWWWWWWWW_W_WW____.........................................
................................................................
................................................................

Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R" Reading Request,
"W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully finishing,
"I" Idle cleanup of worker, "." Open slot with no current process
[root@1 user]#

ammaross · Post by **ammaross** » Mon Mar 16, 2009 7:52 pm

When dealing with Montage view, most browsers cap the number of simultaneous connections to a single server. If you have a large number of cameras, or a low cap, you may not be able to open new windows to ZM nor view all cameras in Montage view. Mine was capped at 7 I believe, which could easily be your 4 cameras, and a couple side-windows with camera views. Try adjusting your FF caps by entering about:config in the address bar and changing these three settings to something that can manage your cameras (I use 24 since I have 16+ cameras on my ZM):

Code: Select all

network.http.max-connections-per-server
network.http.max-persistant-connections-per-proxy
network.http.max-persistant-connections-per-server

c0mputerking · Post by **c0mputerking** » Mon Mar 16, 2009 8:20 pm

I set all to 24 and i think it made things worse seems to work better with an unmodified version of seamonkey. Thinking it might be a Firefox problem as sometimes i cannot connect at all with ff just a white screen but things work ok in seamonkey. i am using the mac version to complicate matters further.

c0mputerking · Post by **c0mputerking** » Mon Mar 16, 2009 8:55 pm

Ok things have frozen with seamonkey now too seems to be an apache problem? even if i try to run apachectl fullstatus apache does not respond ie no output at all, and when i reset httpd it takes way longer that normal to stop like about 1 min wounder what it is doing.

henriquejf · Post by **henriquejf** » Mon May 04, 2009 6:22 pm

In an effort to solve, or even organize related problems, i am asking users experiencing **APACHE HANGS** to double check what has been tried before by other users, so we donÂ´t loose our time and get things working as expected;

I am not even sure that it is a ZM bug, but i am sure that many people have to restart apache from time to time, just as me; One user found it useful to restart apache as a cron job every x minutes, another user proposed to use http://jzmconsole.securitykit.net/ as a workaround for the problem when it happens in the montage view, but i am sure that weÂ´ll have that fixed soon by the community, so we donÂ´t have to give up and try the mentioned workarounds;

apache hanging problems - related, duplicated (or not) topics
==============================================================

** Maxopenfile reached & machine hangs zm problem ?
http://www.zoneminder.com/forums/viewtopic.php?t=13486

** apache not responding
http://www.zoneminder.com/forums/viewtopic.php?t=13322

** Apache instances/responsiveness issue...solved(?)
http://www.zoneminder.com/forums/viewto ... c&start=15

** Zoneminder UNSTABLE & HANGS, needs frequent restarting
http://www.zoneminder.com/forums/viewto ... 1196#51196

** hundreds of httpd processes
http://www.zoneminder.com/forums/viewtopic.php?t=13588

If some experienced programmer could give us a light, it would be so kind !!

Post by **cordel** » Mon May 04, 2009 10:08 pm

See my other response in one of the four threads.

henriquejf · Post by **henriquejf** » Thu May 07, 2009 7:51 pm

I have a simple suggestion of a workaround that works for me: close all instances of your browser and open it again;

Altough it seems to work, you may notice acumulated sock files under /tmp that may indicate that this is a real problem;

I have my experiences and logs documented in
http://www.zoneminder.com/forums/viewtopic.php?p=52491

hope it helps someone;

I still experience these "hangs" (that in fact arenÂ´t apache hanging as you restart your browser and have things working again) with zm 1.24.1 in the latest SVN 2862 (as of may 6th, 2009)

matias.kippes · Post by **matias.kippes** » Wed Sep 09, 2009 7:54 pm

Hello everyone, I'm having the same problem. Probe amend stream.php without positive results.
Also edit the Apache configuration
MaxKeepAliveRequests 100 -----> 500
KeepAliveRequests 15 ---------> 5
This is merely drives the apache connections reach the top faster.

Someone comes up with something else?
Thanks

Maklaut · Post by **Maklaut** » Sun Jun 26, 2011 9:35 am

Possible solution here

ZoneMinder Forums

Apache instances/responsiveness issue...solved

I cannot tell a lie...

top -u apache

Cron Job

duplicated topics

Re: Apache instances/responsiveness issue...solved