ZM 1.25.0 Performance Patch - Completed

[tibetfreedom]

is your performance patch rolled into the new release ?

[knight-of-ni]

is your performance patch rolled into the new release ?

Yes, mastertheknife's performance patch is included, along with a number of other resolved issues.

You can read about what is going on here:
https://github.com/ZoneMinder/ZoneMinde ... state=open

If you know of any unresolved issues then let us know so we can get it added.

Send thanks to Kylejohnson for organizing this on github.

[knight-of-ni]

I also used to use Ubuntu as my desktop OS, but i am much more familiar with RHEL\Centos (especially with yum\rpm over apt\deb) so i am now a Fedora 19 user
You should give that a try, its great!

RHEL\CentOS\Fedora by default ship with selinux enabled, and it pretty much just works. Now and then need to change the default contexts for certain paths with semanage but other than that, its very good and there should be no reason to disable it just for ZM. We will just create a policy module for ZM. A policy module basically defines what our service is allowed to access (assuming it has the correct permissions to access in the first place) and what default access we would like to deny. Pretty much all policy modules in RHEL\CentOS for services at this moment only use the "type" field for controlling access, by having the services run on a special type that can only access what its allowed to access.

The first thing we need is to define types. I thought of least access model where we have type for everything, e.g. zm_exec_t for zm processes and scripts, zm_conf_t for config files such as zm.conf, zm_log_t for files in /var/log/zm, etc, but i believe that wont be possible because the CGI scripts won't be able to access those types because CGI scripts run as httpd_sys_script_exec_t. If we use another type for CGI scripts, apache won't be able to access those CGI files. And if we permit in the policy to allow access from httpd_sys_script_exec_t, every cgi script will be able to access ZM specific stuff, so no point.
So, we can use a minimal approach for now, just to be able to keep selinux in enforcing mode.
Types:
- All executables, including cgi scripts, should their default types. Same goes for logs and everything else.
- Events folder and other zm dynamic content that should be shared between multiple services (such as images/ folder which contains previews for zone editing) - should use public_content_rw_t because i can't think of any easy way of providing access to both ZM and apache.
Access:
- By default, all the default types already have access to other default types.
Ports:
- As far as i remember, ZM doesn't listen on any port, so no need for ports in the policy
Booleans:
- Cant think of any that we might want right now.

Not too difficult it seems

UPDATE: OK, you got me interested in solving this. Looks like someone posted a basic policy here:
http://www.zoneminder.com/wiki/index.ph ... nux_Policy

I added local_zoneminder.te to the rpm package and then configured the rpm %post script to generate and install the selinux policy when the rpm is installed on the target machine. Done!

[mastertheknife]

UPDATE: OK, you got me interested in solving this. Looks like someone posted a basic policy here:
http://www.zoneminder.com/wiki/index.ph ... nux_Policy

I added local_zoneminder.te to the rpm package and then configured the rpm %post script to generate and install the selinux policy when the rpm is installed on the target machine. Done!

Nice job! Had no idea the issue was just few syscalls!
I'm converting my ZM box to CentOS 6.4 right now (Can't stand anything else other than RHEL\CentOS\Fedora after doing RHCE) so i'll test it.

mastertheknife

[tibetfreedom]

I wish to install the FFMPeg library etc is all this as difficult as it was ? or is it easier now ?

[knight-of-ni]

I wish to install the FFMPeg library etc is all this as difficult as it was ? or is it easier now ?

Well, that depends on how you plan to do the installation.
If you wish to make install from source the traditional way, the installation process is the same as it always has been.
ffmpeg will always be tricky due to the rate at which the source code changes.

However, depending on what distro you choose to run on, you have RPM/SRPM options and Ubuntu users have the option of Kylejohnson's puppet module.

[tibetfreedom]

I wish to install the FFMPeg library etc is all this as difficult as it was ? or is it easier now ?

Well, that depends on how you plan to do the installation.
If you wish to make install from source the traditional way, the installation process is the same as it always has been.
ffmpeg will always be tricky due to the rate at which the source code changes.

However, depending on what distro you choose to run on, you have RPM/SRPM options and Ubuntu users have the option of Kylejohnson's puppet module.

Is the puppet module good ?
currently using Mageia which installs FFMPeg by default with zone minder… but it seems unstable with BTTV cards specifically the cloned BTTV 8 chip 8 channel card...

[knight-of-ni]

I wish to install the FFMPeg library etc is all this as difficult as it was ? or is it easier now ?

Well, that depends on how you plan to do the installation.
If you wish to make install from source the traditional way, the installation process is the same as it always has been.
ffmpeg will always be tricky due to the rate at which the source code changes.

However, depending on what distro you choose to run on, you have RPM/SRPM options and Ubuntu users have the option of Kylejohnson's puppet module.

Is the puppet module good ?
currently using Mageia which installs FFMPeg by default with zone minder… but it seems unstable with BTTV cards specifically the cloned BTTV 8 chip 8 channel card...

The puppet module does all the work for you inlcuding create the database, but it is only intended for Ubuntu users. Don't know yet what it does to an existing dB. Testing that is on my to-do list.

Mageia is rpm based, and from what google can tell me, zoneminder may already be in your repository. If that is the case, recommend you download the zoneminder source rpm from Mageia repository, install it to your home folder, and then edit the spec file to point to the latest zoneminder 1.26beta tarball. Then rebuild yourself a new rpm. If you are lucky it will rebuild w/o errors. If you get stuck just let me know, but it would probably be better if you created a new thread.

[jimm01]

Great work on the performance improvements, testing on 1.26 so far indicates the figures obtained are a positive step forward.
It would be great to be able to limit the capture size to a particular size and then scale to whatever in the monitor. Lots of work, but overheads may not be wothwhile,or limit the capture size dependant on
monitor size and give a sensible default. With the present monitor coding your options maybe be limited to what can be done without refactoring large amounts of code.

[mastertheknife]

Great work on the performance improvements, testing on 1.26 so far indicates the figures obtained are a positive step forward.
It would be great to be able to limit the capture size to a particular size and then scale to whatever in the monitor. Lots of work, but overheads may not be wothwhile,or limit the capture size dependant on
monitor size and give a sensible default. With the present monitor coding your options maybe be limited to what can be done without refactoring large amounts of code.

Thank you! But are we done yet? I think a further performance increase will come from AVX2, as it allows integer operations (SSE2's integer instructions under a new prefix actually) to be done on AVX's 256bit registers, which are double the size of SSE's 128bit registers, so its possible to work on 8 32bit pixels/time instead of 4 32bit pixels/time, or 32 greyscale pixels/time instead of 16 greyscale pixels/time. So far only the Intel Haswell processors have AVX2.

But we aren't done yet! There is also AVX-512, which further extends the registers to be 512bit! That would be 16 32bit pixels/time, and 64 greyscale pixels/time! AVX-512 is set to arrive not earlier than 2015.

What do you mean limit the capture size? You mean the size of the captured image? why do that? The only performance increase i can imagine is doing the motion detection work on a smaller image, but if you have to scale that image later on, that performance gain will be gone, and the image will not look as good.

You hit the point! The ZM code is somewhat nasty and difficult to work with. Too much classes inside classes that have no connection to the upper classes whatsoever. I think a rewrite of those areas would be a bless, but no one has the time, its better to focus the time on fixing issues and adding few long-wanted features to ZM.

mastertheknife

[jimm01]

Sounds interesting but I agree that it needs community involvement to truly move forward and get some of the problems solved that make it hard to setup, compile, configure and package for distributions, and helps spread the load and ideas to come up with the best solutions. Unfortunately their are many as always that use the package but do not want to give back anything in return. I think it grew to large for most single developers and their are numerous improvements that can be made but a complete ground up rebuild will be a long way off as the development time alone would stop most in their tracks. Most people want the highest quality of any product but very few are willing to pay the price for it, this is true for FOSS and in the commercial sector.

[mastertheknife]

Sounds interesting but I agree that it needs community involvement to truly move forward and get some of the problems solved that make it hard to setup, compile, configure and package for distributions, and helps spread the load and ideas to come up with the best solutions. Unfortunately their are many as always that use the package but do not want to give back anything in return. I think it grew to large for most single developers and their are numerous improvements that can be made but a complete ground up rebuild will be a long way off as the development time alone would stop most in their tracks. Most people want the highest quality of any product but very few are willing to pay the price for it, this is true for FOSS and in the commercial sector.

So much wisdom.
You are very correct. ZM became big and is not easy to maintain, especially when only one person is doing it. Philip Coombes (user:zoneminder) was the only one working on ZM since 2003 until recently.
With Philip's blessings, the project moved to Github, and there are now 6 developers working on ZM (including me of course). Github makes collaborated development easier, with nice features like pull requests (anyone can contribute to ZM), a good bug tracking system and some other nice stuff.
Many of us don't have a lot of time, just few hours a week to work on ZM, but i believe me and others mostly do it for fun.

Anyone that knows one of these languages: Perl, C++, javascript or PHP and knows basic use of Git can contribute to the project, by sending pull requests. Bugs fixes are always welcome. Improvements and and new features will probably be discussed first in the pull request before merging into the master branch. All that needs to be done is to click the "fork" button, create a new branch to contain the changes that you want to send upstream, create a pull request.
ZoneMinder Github repository: https://github.com/ZoneMinder/ZoneMinder

There has been some progress lately. Few people are working on adding freebsd support, one is working on a new web interface for ZM based on bootstrap (branch modern) and i am working on adding another build system to ZM, as an alternative to the autotools. cmake
http://i.imgur.com/KKqP80q.jpg
http://i.imgur.com/rwHGVCA.jpg

mastertheknife