Shared Memory Config TIPS

[tgutwin]

I have been fighting with a random hang memory problem after an upgrade from a FC4 to FC6 install. It ended up being my shared memory settings. See below for details, but first my system info:

• FC6 with a custom kernel build 2.6.21.5 build for my AMD X2
  3GB (originally 1GB) RAM
  over 700GB disks with JFS filesystem and a logical volume manager
  ZM 1.22.3
  3 Bosch cameras via a ProVideo PV150 - 4 Bt878 port capture card
  Camera settings - 384x240 with FPS=8, alarm FPS=20, buffer =40 frames & 10,20,20 warm pre and post
  Hauppage PVR-350 (mpeg2 vid in/out capture card)

I won't list dmesg or lspci since everything is being picked up OKAY.

The system is also running MythTV (SVN version with the MythTV Zoneminder plugin - Nice Piece Of work!). With both ZM and MythTV I have some significant (but not huge) video capture & streaming requirements. When I was running ZM on FC4 with double the buffer sizes and FPS it was happy. Other reasons forced my to update to FC6. (Video capture drivers for the capture card I am using for MythTV)

Random system hangs and slowdowns were occurring between 1-3 days. /var/log/messages was telling me that I should decreases the buffers or slow down capture.... Which I did. To levels less than what I was running on FC4 and I was still hanging, so I knew it was something else. I followed the FAQ info on setting the Shared memory to no avail.
REMEMBER to restart httpd and ZM after changes to the shared memory settings!

I had tried many, many things but was stumped. I even grabbed and extra 2GB memory to help out, with no luck, UNTIL...
I came across and article about shared memory setting when running DB2. ( http://publib.boulder.ibm.com/infocente ... 008238.htm ). It made some statement that conflicted with those in the FAQ so I gave them a try. The essential difference was that the kernel.shmall setting is NOT in a direct memory setting in KB but in pages of memory. it is Max Pages of memory

For example: If you want to allocate a maximum memory setting to 8GB you have to convert it to the number of pages (or segments).
with a page size of 4096

Code: Select all

kernel.shmall=8000x1024x1024/4096
kernel.shmall=2048000

NOT 8388608000 as would be suggested in the FAQ.

shmmax is the max amount to allocate in one request
this is is an actual memory size (as opposed to pages) set to 4GB

Code: Select all

kernel.shmmax = 4194304000

Can anyone else confirm this is correct??? I don't want to make an update to the Wiki FAQ or a A Shared Memory page on the Wiki until I have confirmed this. I know these values are huge and will adjust them down after some burn -in time.

Here is my /etc/sysctl.conf

Code: Select all

# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
# sysctl.conf(5) for more details.

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0

# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.
kernel.core_uses_pid = 1

# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1

#  The default shared memory limit for SHMMAX is 32 MB and for SHMALL is 2 MB, but it can be changed.
# For example, to allow 128 MB:
# SEE http://publib.boulder.ibm.com/infocenter/db2luw/v9/index.jsp?topic=/com.ibm.db2.udb.uprun.doc/doc/t0008238.htm
# this is the max pages set to 8GB divided by the page size of 4096 = 8000x1024x1024/4096
kernel.shmall = 2048000

# this is is an actual mem size (as opposed to pages) set to 4GB
kernel.shmmax = 4194304000

# reload these with a sysctl -p
# list the mem settings eith a ipcs -l

Here is my interprocess communication status

Code: Select all

[root@dvr tmp]# ipcs -l

------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 4096000
max total shared memory (kbytes) = 8192000
min seg size (bytes) = 1

------ Semaphore Limits --------
max number of arrays = 128
max semaphores per array = 250
max semaphores system wide = 32000
max ops per semop call = 32
semaphore max value = 32767

------ Messages: Limits --------
max queues system wide = 16
max size of message (bytes) = 8192
default max size of queue (bytes) = 16384

NOTE that even though I set the kernel.shmall = 2048000 it is reported (as described and expected) as 8GB

My system has not hung since. So far a full 7 days and not a hint of trouble.

[Kushnirenko]

I can confirm that in Linux
shmmax - is in bytes - max amount of shared memory that can be allocated to one process. Default number is 32MB
shmall - is in pages - total amount of shared memory that can be used by all processes on the computer. Default number is 2097152 pages or 8GB.

When in doubt go to the source . Indeed shm.h in linux kernel says:

Code: Select all

/*
 * SHMMAX, SHMMNI and SHMALL are upper limits are defaults which can
 * be increased by sysctl
 */

#define SHMMAX 0x2000000                 /* max shared seg size (bytes) */
#define SHMMIN 1                         /* min shared seg size (bytes) */
#define SHMMNI 4096                      /* max num of segs system wide */
#define SHMALL (SHMMAX/PAGE_SIZE*(SHMMNI/16)) /* max shm system wide (pages) */
#define SHMSEG SHMMNI                    /* max shared segs per process */

Now what are the recommended values for Zonemider? Here I slightly disagree with FAQ.
My understanding is that each camera will allocate one segment. So shmmax should be calclulated not for all cameras as FAQ suggests but only for one camera. For example camera with 320x240 resolution and GREY colorcode and 40 frames in ring buffer requires:
320x240 x 1byte(Grey) x 40 = 3070200
Similarly RGB24 camera would require 3 times as much
320x240 x 3bytes(RGB) x 40 = 9216000

To be safe you need to increase this number by 10% or something. Indeed ipcs for this setup (one Grey and one RGB24) says that zoneminder allocates slightly larger amount of memory

Code: Select all

0x7a6d2001 2261003    www-data  700        3073188    1  <--- Grey camera
0x7a6d2003 2293772    www-data  700        9217188    1                      <--- RGB24 camera

So to define shmmax pick the most high resolution camera with the biggest ringbuffer and calcluate shmmax in bytes. In the abouve example it should be something like 10MB, or 16MB if you like round numbers. In this case Linux default number 32MB (or 0x2000000) is actually fine

To define shmall take shmmax and multiply by the number of cameras and divide by 4096. To be safe you have to increase this number so that other processes can allocate shared memory too. So in this example it should be:
shmall = 16MB * 2 (cameras) * 2 (safety factor) / 4096 = 16384 (pages). Again Linux default number 2097152 pages or 8GB of memory is also fine.

[jameswilson]

maybe in the setting a recommended setting based on current config, could be displayed?

[nunu]

After troubles and tribulations, here's what I had to do to get my tvip100w-n wireless camera working. Using Zoneminder 1.24.2 with Debian Lenny.

1. First thing is first, memory settings are important, I have a P4 3.ghz machine with 2 gb of memory. I am running virtual box and install Debian Lenny, and dedicated 1gb of memory to this vm. I tried so many settings in /etc/sysctl.conf, then finally found a number that works well, which is kernel.shmmax = 256000000, I did not put in kernel.shmall in the sysctl.conf file. Previously, I had set kernel.shmmax to 128000000, which worked fine for the camera at resolution 320x240, but when I bumped it up to 640x480, it displayed a black screen or a blank screen. Make sure you restart your machine when making any memory changes. I also have a p3 450mhz 768mb of memory, and had the same experience as the p4 3ghz machine. Seems like the kernel.shmmax = 256000000 might be a good number to use.

2. Since this is a wireless camera, interference plays a great deal. I had the camera set outside the house where there was a lot more interference and the camera would stop responding after a while, I had to hard reset it. I also experience lots of camera crashes when tweaking the settings constantly, once you get settings that work for you, you should not need to tweak it much. Basically plan out your wireless channels configurations so that you get the least amount of interference with your neighbors Access Points. I brought the camera inside the house and is looking out of a window, and reconfigured my wireless channels (1,6 or 11) are the ones you should use, the wireless has been pretty flawless, getting a 70-80 percent strength signal on it. What also helped is using the tomato router firmware on my wt54g router, where I could increase the power of the antennae.

3. Here are my camera settings:
640x480
medium compression
auto frame rate
With these settings, I get a decent 5-12 fps

320x240
medium compression
auto frame rate
With these settings, I get a pretty solid 15-27 fps

4. The zoneminder camera settings were quite tricky, but I found something that eluded me for awhile. Here are the key settings below.

Source Tab/remote host path:/video/cgi
Buffers Tab/Image buffer size: 150
Buffers Tab/Warmup frames: 30
Buffers Tab/Stream image replay image buffer: 20000

One of these settings makes the viewing stable, and got rid of memory issue messages in the /var/log/messages. Not sure which one did the trick, and I might have these settings bloated, but I left these alone anyways, it may eat up memory, but at least it's stable, you can tweak these til you find a happy medium.

5. One last thing I did was to tweak the Options/Network setting in Zoneminder. I upped the http_timeout setting to 100000. Probably don't need a number that high, not exactly sure what it really does, but a setting to be experimented with. The combination of the source/buffers/options/network configurations have left my system very stable.

6. I also bought a airlink101 aic250w camera, and is basically the same camera as the tvip100w-n, but the image in lowlight is very fuzzy, but the cost was less than half ($69) of what the tvip100w-n was ($140). Configurations were the exact same, except the frame rate at 640x480 isn't as good as the tvip100w-n. Overall though, it is as stable as the tvip100w-n.

[snaffler]

After three months of trying to get zoneminder to work and tracking down obscure forum posting to obscure website with little tweaks here and there I come to this. I feel like the monkey in 2001 staring at the dark tablet. I don't even know what this forum topic is talking about. Do I just give up and toss my bluecherry board, all of my cameras and PC? There is no way I'm changing memory settings in the kernel without totally screwing everything up.

Can't somebody make a version of linux with all of these settings already done?

[PacoLM]

After three months of trying to get zoneminder to work and tracking down obscure forum posting to obscure website with little tweaks here and there I come to this. I feel like the monkey in 2001 staring at the dark tablet. I don't even know what this forum topic is talking about. Do I just give up and toss my bluecherry board, all of my cameras and PC? There is no way I'm changing memory settings in the kernel without totally screwing everything up.

Can't somebody make a version of linux with all of these settings already done?

Don't give up!. I started to play with Linux some time ago and without experience, just reading and applying the steps, I got my ZM system working. The shared memory settings is a gray area, but depends on your setup. You only have to edit/add two values in a file (/etc/sysctl.conf) and restart the system. I'm talkign about shared memory, I do not know which are your problems with your setup.

Ask for help, this forum it's really helpful!.

PacoLM

[MichaelTiemann]

After troubles and tribulations, here's what I had to do to get my tvip100w-n wireless camera working. Using Zoneminder 1.24.2 with Debian Lenny.

[...]

4. The zoneminder camera settings were quite tricky, but I found something that eluded me for awhile. Here are the key settings below.

Source Tab/remote host path:/video/cgi
Buffers Tab/Image buffer size: 150
Buffers Tab/Warmup frames: 30
Buffers Tab/Stream image replay image buffer: 20000

One of these settings makes the viewing stable, and got rid of memory issue messages in the /var/log/messages. Not sure which one did the trick, and I might have these settings bloated, but I left these alone anyways, it may eat up memory, but at least it's stable, you can tweak these til you find a happy medium.

5. One last thing I did was to tweak the Options/Network setting in Zoneminder. I upped the http_timeout setting to 100000. Probably don't need a number that high, not exactly sure what it really does, but a setting to be experimented with. The combination of the source/buffers/options/network configurations have left my system very stable.

I, too was banging my head against the wall trying to figure out how to increase image buffer size beyond 40 without my cameras going red in the console. I turned on debugging, changed the default 40 to 41, read the logs, and found that Zoneminder was unhappy when it tried to request 41M of shm when only 40M was available. Looking at the source code of the error message, it was clear what was going wrong: once Zoneminder opens a shared memory segment, that's it. What it should do is drop and reallocate the shared memory when one of the parameters related to the shared memory allocation changes. Happily, a restart caused Zoneminder to properly allocate the 41 image buffers I requested, and knowing that, I could easily change the image buffer count to 80, restart Zoneminder again, and now I have lots of pre-roll for my events. I've now dropped my shmmax down to a reasonable number...unreasonable numbers don't help once Zoneminder fixes its size on startup.

[lucy123]

After three months of trying to get zoneminder to work and tracking down obscure forum posting to obscure website with little tweaks here and there I come to this. I feel like the monkey in 2001 staring at the dark tablet. I don't even know what this forum topic is talking about. Do I just give up and toss my bluecherry board, all of my cameras and PC? There is no way I'm changing memory settings in the kernel without totally screwing everything up.

Can't somebody make a version of linux with all of these settings already done?

I feel exactly like this guy. I have tried to solve the problem by following the solutions offered on this board and can't get zoneminder to work....I feel like an idiot trying to figure this out because half of the solutions, I don't even know what they are talking about!
Is there a website that is made for idiots like me?

[bb99]

Lucy, there's no magic here! I used to rely on user RDMelin to supply LiveCD's for the versions as they were upgraded; tear them apart to see how he made them work then do the same for what I wanted. He hasn't posted one since ZMLarch 1.24.2, which worked beautifully! Last build I feel good with is the 1.24.2 against Fedora 12 (wiki: fedora: "the easy way" or seach forums for "flowers for candy"). It's rock solid and works well on older equipment. I do build almost everything that hits the forums but mostly run into problems; it's got to work as posted or you won't hear it from me.

[berton32]

Hey i want to change the memory settings of my new laptop of Dell. But i am unable to do so. Can any one here help me out!!

[bb99]

If you're refering to shared memory settings then: http://www.zoneminder.com/wiki/index.ph ... esolutions