Page 3 of 12
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Wed Jun 22, 2011 10:10 am
by mastertheknife
elusive wrote:xlib_shm (0.6.5 GL) -> no more working while choosing 24 or 32 bits color mode
Hi elusive,
I took a quick look and it seems that nothing can be done. xlib_shm relies on a specific shared memory layout, of which i changed and might change again in the future if new features are added.
Regarding eyezm, I attached a patch to my previous post which adds support for the new formats to zm's mpeg code. eyezm should be working perfectly with BGR24 and 32bit RGB formats after this.
mastertheknife.
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Wed Jun 22, 2011 1:10 pm
by elusive
Thanks for the EyeZM patch and for the xlib_shm check.
I'll give it a try later today and will keep you in touch.
Regarding xlib_shm it's a very bad news as it was my only working alternative, if nothing can be done then I'll have to keep using the original ZoneMinder release unless someone finds a fix regarding ZM Viewer for Linux distributions :-/
If it doesn't bother you too much, could you please let me know what has to be changed from the xlib_shm sources in order to make it working with your patch ?
I mean if the only necessary changes are in the xlib_shm.c file ... I could then update it by myself when you'll make new changes.
Have a nice day.
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Wed Jun 22, 2011 2:06 pm
by elusive
I confirm your EyeZM patch is working fine in 32 bits color mode H264 streaming, well done !
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Wed Jun 22, 2011 5:15 pm
by mastertheknife
elusive wrote:I confirm your EyeZM patch is working fine in 32 bits color mode H264 streaming, well done !
Hi,
Thats good to hear
! I updated the patch in the main post to include this fix and some other tiny fixes and changes.
elusive wrote:Regarding xlib_shm it's a very bad news as it was my only working alternative, if nothing can be done then I'll have to keep using the original ZoneMinder release unless someone finds a fix regarding ZM Viewer for Linux distributions :-/
If it doesn't bother you too much, could you please let me know what has to be changed from the xlib_shm sources in order to make it working with your patch ?
I mean if the only necessary changes are in the xlib_shm.c file ... I could then update it by myself when you'll make new changes.
Have a nice day.
Regarding xlib_shm, technically you only need to replace the structs that belong to ZoneMinder by copying them from zm_monitor.h, its that simple!
Keep in mind xlib_shm only supports the grayscale and RGB24 formats, and will display wrong colour when BGR24 is used (local cameras only) and will not work with the 32bit RGB formats.
I attached a patch for xlib_shm version 0.6.5 especially for you, please also look at it so you can understand what i did\replace.
I can confirm xlib_shm working in RGB24 mode for me.
mastertheknife.
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Thu Jun 23, 2011 2:43 pm
by elusive
Thanks a lot !
xlib_shm is working fine now while using 24 bits color mode, like you said it doesn't work in 32 bits color mode (grey scrambled picture) but the important thing is that it works
The server should be in production next week, once done I'll give you feedbacks regarding the performances but I can already see big improvments regarding the load between original ZM and patched one so I'm quite sure the results will be excellent.
Let's hope your work will be added to trunk and future releases, I guess it will be as soon as developpers will notice the performances it reaches !
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Fri Jun 24, 2011 4:52 pm
by mastertheknife
Hi,
I wrote a SSE2 version of the AlarmedPixels motion detection and i'm now testing it and trying to add it to ZM (not easy to write the complex loops in assembler).
http://i.imgur.com/w6TCe.jpg
This is expected to drop zma's cpu usage even further by ~10%.
mastertheknife
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Mon Jun 27, 2011 11:38 pm
by graphicw
Looking forward to the SSE2 Alarmed Pixels mod. This performance patch works wonders on a server after you remove libjpeg62 and install libjpeg-turbo. The server often runs at 0.00 to 0.04 load with five monitors at 720x480 and at full frame rate. This is with 8 bit grey scale which is all I need anyway. It still averages a load of 0.25 with the monitors set to rbg32 with 32 bit color. This patch is indeed sweet and delivers what it promises.
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Tue Jun 28, 2011 6:19 am
by mastertheknife
graphicw wrote:Looking forward to the SSE2 Alarmed Pixels mod. This performance patch works wonders on a server after you remove libjpeg62 and install libjpeg-turbo. The server often runs at 0.00 to 0.04 load with five monitors at 720x480 and at full frame rate. This is with 8 bit grey scale which is all I need anyway. It still averages a load of 0.25 with the monitors set to rbg32 with 32 bit color. This patch is indeed sweet and delivers what it promises.
Hi,
Thats very good to hear
mastertheknife
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Tue Jun 28, 2011 7:33 am
by mastertheknife
Here are some tests i did with the AlarmedPixels SSE2. Not much improvement over the non-SSE2 code because this part of the motion detection only accounts for about 10% of zma's cpu usage.
6th July 2011 EDIT:
The standard alarmedpixels code is now a lot faster than before, and also faster than the SSE2 AlarmedPixels. I updated the tests below to include this change (pointed by an arrow).
Processor: Intel Core 2 duo E6750 with SSSE3.
ZoneMinder compiled as 64bit with default compiler and linker flags
JPEG library installed: libjpeg-turbo v1.1.0
Monitor settings: 1280x800 JPEG image (File monitor) @ 20 fps, Modect mode.
The default zone covering up the whole picture.
Stock ZM 1.24.4:
Code: Select all
24bit - AlarmedPixels:
13465 apache 20 0 492m 305m 296m R 32 7.7 0:11.77 zmc
13497 apache 20 0 494m 306m 295m S 32 7.7 0:10.39 zma
Unable to test 8bit grayscale because it doesn't work.
Code: Select all
32bit - AlarmedPixels SSE2:
5928 apache 20 0 562m 397m 393m R 29 10.0 0:25.81 zmc
5933 apache 20 0 563m 402m 393m S 14 10.2 0:12.57 zma
32bit - AlarmedPixels:
6552 apache 20 0 562m 397m 393m S 29 10.0 0:09.81 zmc
6585 apache 20 0 563m 402m 393m S 15 10.2 0:04.47 zma ------> is now 12% in patch v1f and newer.
24bit - AlarmedPixels SSE2:
6128 apache 20 0 463m 299m 296m S 29 7.6 0:13.25 zmc
6161 apache 20 0 464m 304m 295m S 17 7.7 0:06.83 zma
24bit - AlarmedPixels:
6353 apache 20 0 463m 299m 296m S 29 7.6 0:12.03 zmc
6389 apache 20 0 464m 304m 295m S 18 7.7 0:06.74 zma ------> is now 15% in patch v1f and newer.
8bit grayscale - AlarmedPixels SSE2:
6922 apache 20 0 266m 104m 100m S 20 2.6 0:04.22 zmc
6927 apache 20 0 267m 106m 100m S 8 2.7 0:01.55 zma
8bit grayscale - AlarmedPixels:
6721 apache 20 0 266m 104m 100m S 20 2.6 0:07.45 zmc
6726 apache 20 0 267m 106m 100m S 9 2.7 0:03.22 zma ------> is now 6% in patch v1f and newer.
mastertheknife
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Wed Jun 29, 2011 1:31 am
by graphicw
Every improvement in CPU usage efficiency is good no matter how small. Looking forward to version 2 of your patch. It is only going to get better from here and I think this is exactly what Zoneminder has needed.
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Wed Jun 29, 2011 9:10 pm
by mastertheknife
graphicw wrote:Every improvement in CPU usage efficiency is good no matter how small. Looking forward to version 2 of your patch. It is only going to get better from here and I think this is exactly what Zoneminder has needed.
Thank you!
I am pretty much reaching the limit, can't cut much more on CPU usage. There is no free lunch, motion detection is still a slow and long process, unless we aren't interested in motion detection, which in that case we wouldn't be using it in the first place..
mastertheknife
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Thu Jun 30, 2011 1:57 pm
by elusive
Hello,
Just to let you know the new v1d patch fails :
Code: Select all
patching file src/zm_event.cpp
Hunk #1 FAILED at 38.
Hunk #2 FAILED at 751.
Hunk #3 FAILED at 1185.
Hunk #4 FAILED at 1196.
Hunk #5 FAILED at 1208.
Hunk #6 FAILED at 1219.
Hunk #7 FAILED at 1240.
Hunk #8 FAILED at 1269.
8 out of 8 hunks FAILED -- saving rejects to file src/zm_event.cpp.rej
I would really like to see it merged to the SVN trunk, any project regarding this ?
Keep up that excellent contribution !
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Thu Jun 30, 2011 3:29 pm
by mastertheknife
elusive wrote:Hello,
Just to let you know the new v1d patch fails
I would really like to see it merged to the SVN trunk, any project regarding this ?
Keep up that excellent contribution !
Hi,
My bad, its fixed now. zm_event.cpp kept giving me problems because it had windows line endings, driving git crazy but i think i just solved it once and for all (dos2unix is a wonderful tool!)
I talked to Phil but from what i understood, its not really possible at this moment because a lot was changed in 1.25.0. However, i will send him fixes for bugs that i fixed and plan on fixing later on.
Thank you!
mastertheknife.
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Thu Jun 30, 2011 5:28 pm
by elusive
You're welcome, I confirm the v1e works fine
I personaly use UltraEdit to edit and convert Linux/DOS files but your soft did the same.
Have a nice evening !
Re: ZM 1.24.4 Performance Patch - Completed
Posted: Mon Jul 04, 2011 12:50 am
by CoYoTe
ppl:
Tonight I have the luck to chat with mastertheknife, and found a solution to one server i have running in a new low profile machine.
His patches makes really an excellent job. In this particular server (Sempron LE1250 w/ 1gb of ram) resources are very low, and this impact directly in overall performance running ZM.
The patches applied make system load to go down drastically. Color space conversion (the main change) and analysis daemons improve a lot with all this work.
Please give a try to this. It's stable enough to set it up to work. And feedback is needed.
Check all the stuff, read carefully the list of changes.
I gonna do this, try the whole suit in 5 servers in "production" with a mix of configurations (different chipsets, micros, amount of RAM, capture boards, ip cams).
For sure, if this stuff pass some tests, can be reviewed to be included in the next release.
I'll post the results.
Greetz
Alejandro
