ZM 1.25.0 Performance Patch - Completed

[tibetfreedom]

You are attempting to use a format that requires libjpeg-turbo, but ZoneMinder was not compiled against libjpeg-turbo. Make sure that libjpeg-turbo's header files are used, and not standard libjpeg's header files.

Please I can't find out how to enable this… I have installed libjpeg-turbo, but have not either compiled gainst it or enabled it in the lib conf where ever that is……

can you do a simple how to on this… ?

I have compiled with the following…

Many Many thanks, your distro is still loads better than the standard non patched…

./configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --with-libarch=lib --with-mysql=/usr --with-ffmpeg=/usr --with-webdir=/var/www/zm --with-cgidir=/usr/lib/cgi-bin --with-webuser=www-data --with-webgroup=www-data --enable-debug=yes --enable-crashtrace=yes ZM_SSL_LIB=openssl CPPFLAGS="-D__STDC_CONSTANT_MACROS"

I will recompile again but no I downloaded the source and compiled.



Did you install libjpeg-turbo from the debian\ubuntu repository? if yes, then thats the issue. Their libjpeg-turbo package only provides the libraries, but not the header files.

mastertheknife

[mastertheknife]

Hi all,

I have recently began working on adding common deinterlacing techniques to ZM and relocating the V4L2_CAPTURE_FIELDS option to be monitor specific.
You can see this here: http://i.imgur.com/sBXhR.jpg
The best deinterlacing technique so far is the four field motion adaptive one. This one compares the current frame with the next frame for motion and only applies linear deinterlacing to motion areas. So far it uses a lot of CPU and is not perfect, but i'm working on improving this.

Here is a picture of a bus captured by my PAL analog camera at 768x576 connected to a bt878 card:


After deinterlacing (Four field motion adaptive):


This should allow us all to use the full resolution of our analog cameras (640x480 for NTSC, 768x576 for PAL).

mastertheknife

[mastertheknife]

tibetfreedom,

What does this command show?

Code: Select all

cat /usr/include/jpeglib.h | grep EXTENSIONS

On my machine:

Code: Select all

kfir@GentooB0X ~ $ cat /usr/include/jpeglib.h | grep EXTENSIONS
#define JCS_EXTENSIONS 1

mastertheknife

[tibetfreedom]

tibetfreedom,

What does this command show?

Code: Select all

cat /usr/include/jpeglib.h | grep EXTENSIONS

On my machine:

Code: Select all

kfir@GentooB0X ~ $ cat /usr/include/jpeglib.h | grep EXTENSIONS
#define JCS_EXTENSIONS 1

mastertheknife

root@CCTV:/home/cctvadmin# cat /usr/include/jpeglib.h | grep EXTENSIONS
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/lib/libjpeg.a /usr/lib/libjpeg.a
ln: creating symbolic link `/usr/lib/libjpeg.a': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jconfig.h /usr/include/jconfig.h
ln: creating symbolic link `/usr/include/jconfig.h': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jerror.h /usr/include/jerror.h
ln: creating symbolic link `/usr/include/jerror.h': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jmorecfg.h /usr/include/jmorecfg.h
ln: creating symbolic link `/usr/include/jmorecfg.h': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jpeglib.h /usr/include/jpeglib.h
ln: creating symbolic link `/usr/include/jpeglib.h': File exists
root@CCTV:/home/cctvadmin# cat /usr/include/jpeglib.h | grep EXTENSIONS
root@CCTV:/home/cctvadmin#

[pok]

mastertheknife, thank your fo your`s Patch.

But i have some problem while use it...
When remote camera work as ffmpeg stream - all ok, but picture some times destroyed.
When remote camera work as rtsp source i have error "swscale does not support the codec format".

Seems trouble in "mFormatContext = rtspThread->getFormatContext();"
I have not more experience in cpp.

As i can see rtsp frame contents send to ffmpeg, so error "swscale does not support the codec format" is not correct.

Please, help me understand what happen in zm_remote_camera_rtsp.cpp.

[mastertheknife]

tibetfreedom,

What does this command show?

Code: Select all

cat /usr/include/jpeglib.h | grep EXTENSIONS

On my machine:

Code: Select all

kfir@GentooB0X ~ $ cat /usr/include/jpeglib.h | grep EXTENSIONS
#define JCS_EXTENSIONS 1

mastertheknife

root@CCTV:/home/cctvadmin# cat /usr/include/jpeglib.h | grep EXTENSIONS
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/lib/libjpeg.a /usr/lib/libjpeg.a
ln: creating symbolic link `/usr/lib/libjpeg.a': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jconfig.h /usr/include/jconfig.h
ln: creating symbolic link `/usr/include/jconfig.h': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jerror.h /usr/include/jerror.h
ln: creating symbolic link `/usr/include/jerror.h': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jmorecfg.h /usr/include/jmorecfg.h
ln: creating symbolic link `/usr/include/jmorecfg.h': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jpeglib.h /usr/include/jpeglib.h
ln: creating symbolic link `/usr/include/jpeglib.h': File exists
root@CCTV:/home/cctvadmin# cat /usr/include/jpeglib.h | grep EXTENSIONS
root@CCTV:/home/cctvadmin#

The problem is that you are using stock libjpeg's header files, instead of libjpeg-turbo's header files which include extensions.

What does this show?

Code: Select all

cat /opt/libjpeg-turbo/include/jpeglib.h | grep EXTENSIONS

If it shows the correct output, then do this:

Code: Select all

rm /usr/lib/libjpeg.a
rm /usr/include/jconfig.h
rm /usr/include/jerror.h
rm /usr/include/jmorecfg.h
rm /usr/include/jpeglib.h

Code: Select all

ln -s /opt/libjpeg-turbo/lib/libjpeg.a /usr/lib/libjpeg.a
ln -s /opt/libjpeg-turbo/include/jconfig.h /usr/include/jconfig.h
ln -s /opt/libjpeg-turbo/include/jerror.h /usr/include/jerror.h
ln -s /opt/libjpeg-turbo/include/jmorecfg.h /usr/include/jmorecfg.h
ln -s /opt/libjpeg-turbo/include/jpeglib.h /usr/include/jpeglib.h

Then recompile ZM (make) and reinstall ZM (make install) and hopefully it will work this time.
mastertheknife

[beerygaz]

Firstly I'd like to applaud your work and innovation. I'm really looking forward to the interlacing code, especially.

I compiled your source but can only get my cameras (local via bt878) to work in greyscale. If I select BGR24 (and 24-bit color) or BGR32 I get the following errors:

Code: Select all

Mar  3 08:28:37 dvr zms[8835]: ERR [Bogus input colorspace]
Mar  3 08:28:37 dvr abrt[8838]: saved core dump of pid 8835 (/data/www/cgi-bin/nph-zms) to /var/spool/abrt/ccpp-2012-03-03-08:28:37-8835.new/coredump (94789632bytes)
Mar  3 08:28:37 dvr abrtd: Directory 'ccpp-2012-03-03-08:28:37-8835' creation detected
Mar  3 08:28:38 dvr abrtd: Executable '/data/www/cgi-bin/nph-zms' doesn't belong to any package
Mar  3 08:28:38 dvr abrtd: Corrupted or bad dump /var/spool/abrt/ccpp-2012-03-03-08:28:37-8835 (res:2), deleting
Mar  3 08:28:39 dvr web_php[18346]: ERR [socket_sendto( /tmp/zm/zms-903499s.sock ) failed: Connection refused]
Mar  3 08:28:39 dvr auditd[1501]: Audit daemon rotating log files
Mar  3 08:28:42 dvr web_php[18345]: ERR [socket_sendto( /tmp/zm/zms-903499s.sock ) failed: Connection refused]

In addition, it seems zmtrigger.pl is not working. My zmtrigger.log shows it receiving events, but nothing happens in ZM (I send an on+30 event and the monitor remains idle).

Can you shed any light on what's going wrong please? I'd love to get my colour images back.

Gaz

[mastertheknife]

beerygaz,

Thank you!

The reason you are seeing "Bogus input colorspace" is because standard libjpeg is being loaded, instead of libjpeg-turbo which is required for BGR24 or 32bit.
About zmtrigger.pl, thanks for reporting, i'll look into this.

mastertheknife

[mastertheknife]

mastertheknife, thank your fo your`s Patch.

But i have some problem while use it...
When remote camera work as ffmpeg stream - all ok, but picture some times destroyed.
When remote camera work as rtsp source i have error "swscale does not support the codec format".

Seems trouble in "mFormatContext = rtspThread->getFormatContext();"
I have not more experience in cpp.

As i can see rtsp frame contents send to ffmpeg, so error "swscale does not support the codec format" is not correct.

Please, help me understand what happen in zm_remote_camera_rtsp.cpp.

swscale is being used by ZoneMinder to convert from certain formats to formats that ZM can understand and work with (greyscale and RGB).

mastertheknife

[beerygaz]

beerygaz,

Thank you!

The reason you are seeing "Bogus input colorspace" is because standard libjpeg is being loaded, instead of libjpeg-turbo which is required for BGR24 or 32bit.
About zmtrigger.pl, thanks for reporting, i'll look into this.

mastertheknife

Cn you tell me how to comile libjpeg-turbo then please. I've followed as much as I can from this thread.

Code: Select all

[root@dvr zm]# cat /opt/libjpeg-turbo/include/jpeglib.h | grep EXTENSIONS
#define JCS_EXTENSIONS 1
#define JCS_ALPHA_EXTENSIONS 1

I followed the previous post of removing the existing libjepg files and creating the symbolic links. Then I re-ran configure and make / make install - but I'm still getting the same [Bogus input colorspace] error.

Am I missing a step?

Gaz

[mastertheknife]

These commands simply take care of libjpeg-turbo's header files and the static library.
The correct libjpeg.so still needs to be loaded. There are few days to do this, the easiest one is to replace the standard libjpeg.so with the one from libjpeg-turbo's by copying /opt/libjpeg-turbo/lib into /usr/lib but this affects all applications on the system.

mastertheknife

[rtindle]

I'm getting the following error during make:

Code: Select all

g++ -DHAVE_CONFIG_H -I. -I..  -I/usr/include -I/usr/include -Wall -Wno-sign-compare -finline-functions -I/usr/include   -D__STDC_CONSTANT_MACROS -DZM_STRIP_SSE -MT zm_ffmpeg_camera.o -MD -MP -MF .deps/zm_ffmpeg_camera.Tpo -c -o zm_ffmpeg_camera.o zm_ffmpeg_camera.cpp
zm_ffmpeg_camera.cpp: In member function âvirtual int FfmpegCamera::Capture(Image&)â:
zm_ffmpeg_camera.cpp:219: error: âSWS_CPU_CAPS_SSE2â was not declared in this scope
make[2]: *** [zm_ffmpeg_camera.o] Error 1
make[2]: Leaving directory `/home/rtindle/mastertheknife-ZoneMinder-kfir-8d30497/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/rtindle/mastertheknife-ZoneMinder-kfir-8d30497'
make: *** [all] Error 2

This is on a CentOS 6.2 ESXi VM; ffmpeg-0.6.5-1.el6.rf.i686/ffmpeg-devel-0.6.5-1.el6.rf.i686 along with libjpeg-turbo-1.2.0-20120209.i386 are installed.

Code: Select all

flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss constant_tsc pebs bts tsc_reliable nonstop_tsc unfair_spinlock pni hypervisoru

[mastertheknife]

pok,
I have updated the patch and the error now includes the unsupported format in the error. So please re-download the source and this should make troubleshooting easier.

rtindle,
In the update i have also added a workaround against this error. Please re-download the source and try again.

mastertheknife

[rtindle]

rtindle,
In the update i have also added a workaround against this error. Please re-download the source and try again.

This works great; compiled and now running; thank you for all the help you've done here; this patch is very useful and already I can see an improvement in my setups performance.

One thing I had to do after the fact was fix the jpeg corruption issues my cameras (Foscam) are sending out. The fix is here:

http://www.zoneminder.com/forums/viewto ... 440#p73440

The corruption isn't a problem; but the spamming of the logs makes it hard to see real issues.

[beerygaz]

These commands simply take care of libjpeg-turbo's header files and the static library.
The correct libjpeg.so still needs to be loaded. There are few days to do this, the easiest one is to replace the standard libjpeg.so with the one from libjpeg-turbo's by copying /opt/libjpeg-turbo/lib into /usr/lib but this affects all applications on the system.

mastertheknife

I just wanted to say thanks for the assistance. I have your patch up and running and ZM is working better than it ever has. RTSP streaming is working, fps rates are up, CPU is down. Great stuff.

Any ideas on why zmtrigger.pl is still not working as expected though?

Gaz