ZM 1.25.0 Performance Patch - Completed

[mastertheknife]

xlib_shm (0.6.5 GL) -> no more working while choosing 24 or 32 bits color mode

Hi elusive,

I took a quick look and it seems that nothing can be done. xlib_shm relies on a specific shared memory layout, of which i changed and might change again in the future if new features are added.

Regarding eyezm, I attached a patch to my previous post which adds support for the new formats to zm's mpeg code. eyezm should be working perfectly with BGR24 and 32bit RGB formats after this.

mastertheknife.

[elusive]

Thanks for the EyeZM patch and for the xlib_shm check.

I'll give it a try later today and will keep you in touch.

Regarding xlib_shm it's a very bad news as it was my only working alternative, if nothing can be done then I'll have to keep using the original ZoneMinder release unless someone finds a fix regarding ZM Viewer for Linux distributions :-/

If it doesn't bother you too much, could you please let me know what has to be changed from the xlib_shm sources in order to make it working with your patch ?

I mean if the only necessary changes are in the xlib_shm.c file ... I could then update it by myself when you'll make new changes.

Have a nice day.

[elusive]

I confirm your EyeZM patch is working fine in 32 bits color mode H264 streaming, well done !

[mastertheknife]

I confirm your EyeZM patch is working fine in 32 bits color mode H264 streaming, well done !

Hi,

Thats good to hear ! I updated the patch in the main post to include this fix and some other tiny fixes and changes.

Regarding xlib_shm it's a very bad news as it was my only working alternative, if nothing can be done then I'll have to keep using the original ZoneMinder release unless someone finds a fix regarding ZM Viewer for Linux distributions :-/

If it doesn't bother you too much, could you please let me know what has to be changed from the xlib_shm sources in order to make it working with your patch ?

I mean if the only necessary changes are in the xlib_shm.c file ... I could then update it by myself when you'll make new changes.

Have a nice day.

Regarding xlib_shm, technically you only need to replace the structs that belong to ZoneMinder by copying them from zm_monitor.h, its that simple!
Keep in mind xlib_shm only supports the grayscale and RGB24 formats, and will display wrong colour when BGR24 is used (local cameras only) and will not work with the 32bit RGB formats.

I attached a patch for xlib_shm version 0.6.5 especially for you, please also look at it so you can understand what i did\replace.

xlib_shm_patch.patch.zip

(1.04 KiB) Downloaded 265 times

I can confirm xlib_shm working in RGB24 mode for me.

mastertheknife.

[elusive]

Thanks a lot !

xlib_shm is working fine now while using 24 bits color mode, like you said it doesn't work in 32 bits color mode (grey scrambled picture) but the important thing is that it works

The server should be in production next week, once done I'll give you feedbacks regarding the performances but I can already see big improvments regarding the load between original ZM and patched one so I'm quite sure the results will be excellent.

Let's hope your work will be added to trunk and future releases, I guess it will be as soon as developpers will notice the performances it reaches !

[mastertheknife]

Hi,
I wrote a SSE2 version of the AlarmedPixels motion detection and i'm now testing it and trying to add it to ZM (not easy to write the complex loops in assembler).
http://i.imgur.com/w6TCe.jpg
This is expected to drop zma's cpu usage even further by ~10%.

mastertheknife

[graphicw]

Looking forward to the SSE2 Alarmed Pixels mod. This performance patch works wonders on a server after you remove libjpeg62 and install libjpeg-turbo. The server often runs at 0.00 to 0.04 load with five monitors at 720x480 and at full frame rate. This is with 8 bit grey scale which is all I need anyway. It still averages a load of 0.25 with the monitors set to rbg32 with 32 bit color. This patch is indeed sweet and delivers what it promises.

[mastertheknife]

Looking forward to the SSE2 Alarmed Pixels mod. This performance patch works wonders on a server after you remove libjpeg62 and install libjpeg-turbo. The server often runs at 0.00 to 0.04 load with five monitors at 720x480 and at full frame rate. This is with 8 bit grey scale which is all I need anyway. It still averages a load of 0.25 with the monitors set to rbg32 with 32 bit color. This patch is indeed sweet and delivers what it promises.

Hi,
Thats very good to hear

mastertheknife

[mastertheknife]

Here are some tests i did with the AlarmedPixels SSE2. Not much improvement over the non-SSE2 code because this part of the motion detection only accounts for about 10% of zma's cpu usage.

6th July 2011 EDIT:
The standard alarmedpixels code is now a lot faster than before, and also faster than the SSE2 AlarmedPixels. I updated the tests below to include this change (pointed by an arrow).

Processor: Intel Core 2 duo E6750 with SSSE3.
ZoneMinder compiled as 64bit with default compiler and linker flags
JPEG library installed: libjpeg-turbo v1.1.0
Monitor settings: 1280x800 JPEG image (File monitor) @ 20 fps, Modect mode.
The default zone covering up the whole picture.

Stock ZM 1.24.4:

Code: Select all

24bit - AlarmedPixels:
 13465 apache    20   0  492m 305m 296m R   32  7.7   0:11.77 zmc                                                                                            
 13497 apache    20   0  494m 306m 295m S   32  7.7   0:10.39 zma

Unable to test 8bit grayscale because it doesn't work.

Patch:

Code: Select all

32bit - AlarmedPixels SSE2:
 5928 apache    20   0  562m 397m 393m R   29 10.0   0:25.81 zmc                                                                                            
 5933 apache    20   0  563m 402m 393m S   14 10.2   0:12.57 zma

32bit - AlarmedPixels:
 6552 apache    20   0  562m 397m 393m S   29 10.0   0:09.81 zmc                                                                                            
 6585 apache    20   0  563m 402m 393m S   15 10.2   0:04.47 zma ------> is now 12% in patch v1f and newer.

24bit - AlarmedPixels SSE2:
 6128 apache    20   0  463m 299m 296m S   29  7.6   0:13.25 zmc                                                                                            
 6161 apache    20   0  464m 304m 295m S   17  7.7   0:06.83 zma

24bit - AlarmedPixels:
 6353 apache    20   0  463m 299m 296m S   29  7.6   0:12.03 zmc                                                                                            
 6389 apache    20   0  464m 304m 295m S   18  7.7   0:06.74 zma ------> is now 15% in patch v1f and newer.

8bit grayscale - AlarmedPixels SSE2:
 6922 apache    20   0  266m 104m 100m S   20  2.6   0:04.22 zmc                                                                                            
 6927 apache    20   0  267m 106m 100m S    8  2.7   0:01.55 zma

8bit grayscale - AlarmedPixels:
 6721 apache    20   0  266m 104m 100m S   20  2.6   0:07.45 zmc                                                                                            
 6726 apache    20   0  267m 106m 100m S    9  2.7   0:03.22 zma ------> is now 6% in patch v1f and newer.

mastertheknife

[graphicw]

Every improvement in CPU usage efficiency is good no matter how small. Looking forward to version 2 of your patch. It is only going to get better from here and I think this is exactly what Zoneminder has needed.

[mastertheknife]

Every improvement in CPU usage efficiency is good no matter how small. Looking forward to version 2 of your patch. It is only going to get better from here and I think this is exactly what Zoneminder has needed.

Thank you!
I am pretty much reaching the limit, can't cut much more on CPU usage. There is no free lunch, motion detection is still a slow and long process, unless we aren't interested in motion detection, which in that case we wouldn't be using it in the first place..

mastertheknife

[elusive]

Hello,

Just to let you know the new v1d patch fails :

Code: Select all

patching file src/zm_event.cpp
Hunk #1 FAILED at 38.
Hunk #2 FAILED at 751.
Hunk #3 FAILED at 1185.
Hunk #4 FAILED at 1196.
Hunk #5 FAILED at 1208.
Hunk #6 FAILED at 1219.
Hunk #7 FAILED at 1240.
Hunk #8 FAILED at 1269.
8 out of 8 hunks FAILED -- saving rejects to file src/zm_event.cpp.rej

I would really like to see it merged to the SVN trunk, any project regarding this ?

Keep up that excellent contribution !

[mastertheknife]

Hello,

Just to let you know the new v1d patch fails

I would really like to see it merged to the SVN trunk, any project regarding this ?

Keep up that excellent contribution !

Hi,

My bad, its fixed now. zm_event.cpp kept giving me problems because it had windows line endings, driving git crazy but i think i just solved it once and for all (dos2unix is a wonderful tool!)

I talked to Phil but from what i understood, its not really possible at this moment because a lot was changed in 1.25.0. However, i will send him fixes for bugs that i fixed and plan on fixing later on.

Thank you!
mastertheknife.

[elusive]

You're welcome, I confirm the v1e works fine

I personaly use UltraEdit to edit and convert Linux/DOS files but your soft did the same.

Have a nice evening !

[CoYoTe]

ppl:

Tonight I have the luck to chat with mastertheknife, and found a solution to one server i have running in a new low profile machine.

His patches makes really an excellent job. In this particular server (Sempron LE1250 w/ 1gb of ram) resources are very low, and this impact directly in overall performance running ZM.

The patches applied make system load to go down drastically. Color space conversion (the main change) and analysis daemons improve a lot with all this work.

Please give a try to this. It's stable enough to set it up to work. And feedback is needed.

Check all the stuff, read carefully the list of changes.

I gonna do this, try the whole suit in 5 servers in "production" with a mix of configurations (different chipsets, micros, amount of RAM, capture boards, ip cams).

For sure, if this stuff pass some tests, can be reviewed to be included in the next release.

I'll post the results.

Greetz
Alejandro