Daily alarm events video script

If you've made a patch to quick fix a bug or to add a new feature not yet in the main tree then post it here so others can try it out.
Andre81
Posts: 14
Joined: Thu Jun 25, 2009 10:25 am

Re: Daily alarm events video script

Post by Andre81 »

ddalton wrote:I have tried all the sort commands but I cannot prevent the error below if you look at 104 and 105?

/var/www/html/zm/events/1/11/09/29/11/50/00/1039-capture.jpg
/var/www/html/zm/events/1/11/09/29/11/50/00/104-capture.jpg
/var/www/html/zm/events/1/11/09/29/11/50/00/1040-capture.jpg
/var/www/html/zm/events/1/11/09/29/11/50/00/1041-capture.jpg
/var/www/html/zm/events/1/11/09/29/11/50/00/1042-capture.jpg
/var/www/html/zm/events/1/11/09/29/11/50/00/1043-capture.jpg
/var/www/html/zm/events/1/11/09/29/11/50/00/1044-capture.jpg
/var/www/html/zm/events/1/11/09/29/11/50/00/1045-capture.jpg
/var/www/html/zm/events/1/11/09/29/11/50/00/1046-capture.jpg
/var/www/html/zm/events/1/11/09/29/11/50/00/1047-capture.jpg
/var/www/html/zm/events/1/11/09/29/11/50/00/1048-capture.jpg
/var/www/html/zm/events/1/11/09/29/11/50/00/1049-capture.jpg
/var/www/html/zm/events/1/11/09/29/11/50/00/105-capture.jpg
/var/www/html/zm/events/1/11/09/29/11/50/00/1050-capture.jpg
/var/www/html/zm/events/1/11/09/29/11/50/00/1051-capture.jpg
Hi @ all,

I have the same problem, I have tried all of sort option in file, but the problem is still there.

Seems that, if you use ls command following sort -n command the sorting works properly, but when you use absolute path with find command didn't work.
Probably sort command isn't enough powerful in order to work with a string row...

Any help are appreciated

A.
vraa
Posts: 31
Joined: Mon May 23, 2011 6:28 am

Re: Daily alarm events video script

Post by vraa »

Any update on the 1000+ frame issue?
vraa
Posts: 31
Joined: Mon May 23, 2011 6:28 am

Re: Daily alarm events video script

Post by vraa »

vraa wrote:Any update on the 1000+ frame issue?
I will post my results in a bit, if I forgot someone remind me

But I figured out a way to get around the 1000 frame issue, I use "sort -V"

I am working on speeding up the video generation, It seems mencoder or libavc only uses 1 core for mpeg4 encoding, even with threads=4 passed as an option, I see the threads in htop, but cpu usage isnt maxed out

I'll perhaps try h264

Or better yet, rewrite the script to create all the sorted.lists from the monitors, and then use bash parallel to spawn off as many mencoder threads as we have cores on the box

I do not think it's possible to speed up the find command.

Code: Select all

#!/bin/bash

# Use this script for creating alarm video-files with zoneminder 1.25.0
# by Nightcrawler
# UPDATE the archive directory (YOURARCHIVEDIRECTORY)

#make temp directory
mkdir -p /tmp/alarmvideos

# Enumerate existing ZoneMinder Monitors
cameraname=(`ls /usr/share/zoneminder/events/ | grep '[a-zA-Z]'`)
cameraList=(`ls /usr/share/zoneminder/events/ | grep '[0-9]'`)
cameraNum=${#cameraList[@]}
eventsFolder=/usr/share/zoneminder/events/

echo Number of detected ZoneMinder Monitors: ${#cameraList[@]}

# Start working through all detected ZoneMinder Monitors
	# The camera's get the name from the directory symlink on alphanumeric order!
	# Alter every parameter entry BELOW THIS LINE called {cameraname[$i]} to ${cameralist[$i]} if you don't want this.

 for (( i=0; i<${cameraNum}; i++ ));
 do
  echo Doing ZoneMinder Monitor: ${cameraname[$i]}

# Get a list of images for the last 24 hours from each of the detected ZoneMinder Monitors.
  find   $eventsFolder${cameraList[$i]}  -mtime -3 -name \*capture.jpg > /tmp/alarmvideos/${cameraname[$i]}.list

# Sort output file for a consistent movie
sort /tmp/alarmvideos/${cameraname[$i]}.list -V -o /tmp/alarmvideos/${cameraname[$i]}-sorted.list

# use local time settings to order the events right.
#LC_ALL=C sort /tmp/alarmvideos/${cameraname[$i]}.list -d -o /tmp/alarmvideos/${cameraname[$i]}-sorted-time.list
	
# Encode each 24hr events into a movie Mpeg4 (edit with&hight (w=640:h=480) for your own cam settings)
# to edit the framespeed edit -FPS=5 and -OFPS 5 in the syntax

mencoder mf://@/tmp/alarmvideos/${cameraname[$i]}-sorted.list  -mf w=640:h=480:fps=15:type=jpg -ovc lavc -lavcopts threads=4:vcodec=mpeg4 -ofps 15 -oac copy -o /tmp/alarmvideos/`date +%d`-${cameraname[$i]}.avi

echo video files ready, now moving to archive


# ------------------- start moving files ------------------

# create directory for year and month (if it's nog existing)
#mkdir -p /YOURARCHIVEDIRECTORY/images/videos/`date +%Y`/`date +%m`/

# Find avi's with actual events-move to new dir
#cd /tmp/alarmvideos
#find *.avi -size +5k -exec mv {} /YOURARCHIVEDIRECTORY/images/videos/`date +%Y`/`date +%m` \;

# Cleanup temp files
#rm -rf /tmp/alarmvideos/*.list
#rm -rf /tmp/alarmvideos/*.avi
	
	# Cleanup after 1 year archive (optional feature!)
#find /YOURARCHIVEDIRECTORY/images/videos* -mtime +365 -exec rm {} \;

# ------------------- end moving files ------------------

echo DONE!

  done
I have commented a lot of the original code out for my testing
I changed my location for events to fit Ubuntu style
I changed mtime to 3, for 3 days rather than 1 day
I changed the sort command and added -V for --version-sort
I changed my camera fps to 15, and added threads=4
vraa
Posts: 31
Joined: Mon May 23, 2011 6:28 am

Re: Daily alarm events video script

Post by vraa »

Turned it into x264, and it still works

Code: Select all

#!/bin/bash

# Use this script for creating alarm video-files with zoneminder 1.25.0
# by Nightcrawler
# UPDATE the archive directory (YOURARCHIVEDIRECTORY)

#make temp directory
mkdir -p /tmp/alarmvideos

# Enumerate existing ZoneMinder Monitors
cameraname=(`ls /usr/share/zoneminder/events/ | grep '[a-zA-Z]'`)
cameraList=(`ls /usr/share/zoneminder/events/ | grep '[0-9]'`)
cameraNum=${#cameraList[@]}
eventsFolder=/usr/share/zoneminder/events/

echo Number of detected ZoneMinder Monitors: ${#cameraList[@]}

# Start working through all detected ZoneMinder Monitors
	# The camera's get the name from the directory symlink on alphanumeric order!
	# Alter every parameter entry BELOW THIS LINE called {cameraname[$i]} to ${cameralist[$i]} if you don't want this.

 for (( i=0; i<${cameraNum}; i++ ));
 do
  echo Doing ZoneMinder Monitor: ${cameraname[$i]}

# Get a list of images for the last 24 hours from each of the detected ZoneMinder Monitors.
  find   $eventsFolder${cameraList[$i]}  -mtime -1 -name \*capture.jpg > /tmp/alarmvideos/${cameraname[$i]}.list

# Sort output file for a consistent movie
sort /tmp/alarmvideos/${cameraname[$i]}.list -V -o /tmp/alarmvideos/${cameraname[$i]}-sorted.list

# use local time settings to order the events right.
#LC_ALL=C sort /tmp/alarmvideos/${cameraname[$i]}.list -d -o /tmp/alarmvideos/${cameraname[$i]}-sorted-time.list
	
# Encode each 24hr events into a movie Mpeg4 (edit with&hight (w=640:h=480) for your own cam settings)
# to edit the framespeed edit -FPS=5 and -OFPS 5 in the syntax

#mencoder mf://@/tmp/alarmvideos/${cameraname[$i]}-sorted.list  -mf w=640:h=480:fps=15:type=jpg -ovc lavc -lavcopts threads=4:vcodec=mpeg4 -ofps 15 -oac copy -o /tmp/alarmvideos/`date +%d`-${cameraname[$i]}.avi
mencoder mf://@/tmp/alarmvideos/${cameraname[$i]}-sorted.list -mf w=640:h=480:fps=15:type=jpg \
	-ovc x264 -x264encopts turbo:bitrate=900:subq=4:bframes=2:b_pyramid=normal:weight_b:threads=auto \
	-oac copy -o /tmp/alarmvideos/`date +%d`-${cameraname[$i]}.avi


echo video files ready, now moving to archive


# ------------------- start moving files ------------------

# create directory for year and month (if it's nog existing)
#mkdir -p /YOURARCHIVEDIRECTORY/images/videos/`date +%Y`/`date +%m`/

# Find avi's with actual events-move to new dir
#cd /tmp/alarmvideos
#find *.avi -size +5k -exec mv {} /YOURARCHIVEDIRECTORY/images/videos/`date +%Y`/`date +%m` \;

# Cleanup temp files
#rm -rf /tmp/alarmvideos/*.list
#rm -rf /tmp/alarmvideos/*.avi
	
	# Cleanup after 1 year archive (optional feature!)
#find /YOURARCHIVEDIRECTORY/images/videos* -mtime +365 -exec rm {} \;

# ------------------- end moving files ------------------

echo DONE!

  done
User avatar
nightcrawler
Posts: 71
Joined: Fri Aug 12, 2011 9:54 am
Location: the netherlands

Re: Daily alarm events video script

Post by nightcrawler »

Hi vraa,

Big thanks for finding an simple sollution for the 1000 frames sorting. I will use that :)
And the X264 version is an nice recode option :) I also have an .FLV version for integrating with and website. but X264 is an good alternative for mpeg coding.

nice job!
Zoneminder @Ubuntu 11 server, ZM 1.25.0 , FFMPEG, 4 Analoge cameras, 1.6ghz P4mobile, 2GB ram, 60GB-HDD.
I also have an (homeseer) domotica system up-and-running with touchscreens,light controllers,weather forecast etc.
vraa
Posts: 31
Joined: Mon May 23, 2011 6:28 am

Re: Daily alarm events video script

Post by vraa »

nightcrawler wrote:Hi vraa,

Big thanks for finding an simple sollution for the 1000 frames sorting. I will use that :)
And the X264 version is an nice recode option :) I also have an .FLV version for integrating with and website. but X264 is an good alternative for mpeg coding.

nice job!
Thanks

Actually I re did the script, I'll post results asap

I made the script generate the list of jpgs first, because otherwise things get a little messed up because video generation takes more than 1 hr.

This way the .list files are created within a few minutes of each other, rather than ~6 hrs from one another
vraa
Posts: 31
Joined: Mon May 23, 2011 6:28 am

Re: Daily alarm events video script

Post by vraa »

I am having some trouble with my number of detected monitors

Unfortunately, my monitors are named as follows:
Monitor-1
Monitor-2
Monitor-3
Monitor-5

This causes this line

Code: Select all

cameraList=(`ls /usr/share/zoneminder/events/ | grep '[0-9]'`)
To return

Code: Select all

vraa@vkcsvr-zm49264:/tmp/alarmvideos$ ls /usr/share/zoneminder/events/ | grep '[0-9]'
1
2
3
5
Monitor-1
Monitor-2
Monitor-3
Monitor-5
vraa@vkcsvr-zm49264:/tmp/alarmvideos$
So when this is run

Code: Select all

cameraNum=${#cameraList[@]}
It returns 8 instead of 4.
vraa
Posts: 31
Joined: Mon May 23, 2011 6:28 am

Re: Daily alarm events video script

Post by vraa »

I changed the way the regular expressions work when enumerating the camera list and names

Here is the most recent example of my code so far, I'll be updating it in a little bit again after I figure out how to create the sorted .list files en masse before starting the encoding en masse.

Code: Select all

#!/bin/bash

# Use this script for creating alarm video-files with zoneminder 1.25.0
# by Nightcrawler
# edited by vraa (ubuntu friendly, zoneminder 1.24 friendly, h264 friendly, 1000+ frame friendly)

#make temp directory
mkdir -p /tmp/alarmvideos

# Enumerate existing ZoneMinder Monitors
cameraname=(`ls /usr/share/zoneminder/events/ | grep '[a-zA-Z]'`)
cameraList=(`ls /usr/share/zoneminder/events/ | grep '^[0-9]'`)
cameraNum=${#cameraList[@]}
eventsFolder=/usr/share/zoneminder/events/

echo Number of detected ZoneMinder Monitors: ${#cameraList[@]}

for (( i=0; i<${cameraNum}; i++ ));
do
	echo Doing ZoneMinder Monitor: ${cameraname[$i]}

	find $eventsFolder${cameraList[$i]} -mtime -1 -name \*capture.jpg > /tmp/alarmvideos/${cameraname[$i]}.list

	sort /tmp/alarmvideos/${cameraname[$i]}.list -V -o /tmp/alarmvideos/${cameraname[$i]}-sorted.list

	mencoder mf://@/tmp/alarmvideos/${cameraname[$i]}-sorted.list -mf w=640:h=480:fps=15:type=jpg \
		-ovc lavc -lavcopts vbitrate=600:turbo:threads=4:vcodec=mpeg4:vmax_b_frames=0:vme=4:mbd=1:autoaspect:dia=1:vb_strategy=0:predia=1:last_pred=0 \
#		-ovc xvid -xvidencopts bitrate=600:turbo:nochroma_me:notrellis:max_bframes=0:vhq=0:threads=4 \
#		-ovc x264 -x264encopts bitrate=700:subq=4:bframes=2:b_pyramid=normal:weight_b:threads=auto \
		-oac copy \
		-o /tmp/alarmvideos/`date +%d`-${cameraname[$i]}.avi

# create directory for year and month (if it's not existing)
#mkdir -p /YOURARCHIVEDIRECTORY/images/videos/`date +%Y`/`date +%m`/

# Find avi's with actual events-move to new dir
#cd /tmp/alarmvideos
#find *.avi -size +5k -exec mv {} /YOURARCHIVEDIRECTORY/images/videos/`date +%Y`/`date +%m` \;

# Cleanup temp files
#rm -rf /tmp/alarmvideos/*.list
#rm -rf /tmp/alarmvideos/*.avi
	
# Cleanup after 1 year archive 
#find /YOURARCHIVEDIRECTORY/images/videos* -mtime +365 -exec rm {} \;

	echo DONE!
done
vraa
Posts: 31
Joined: Mon May 23, 2011 6:28 am

Re: Daily alarm events video script

Post by vraa »

I haven't tested this out, just posting work in progress

I have split up the list generation into one for loop and the video generation into another for loop

Code: Select all

#!/bin/bash

# use this script for creating alarm video-files with zoneminder 1.25.0
# by nightcrawler
# edited by vraa (ubuntu friendly, zoneminder 1.24 friendly, h264 friendly, 1000+ frame friendly)

#make temp directory
mkdir -p /tmp/alarmvideos

# enumerate existing zoneminder monitors
cameraname=(`ls /usr/share/zoneminder/events/ | grep '[a-za-z]'`)
cameralist=(`ls /usr/share/zoneminder/events/ | grep '^[0-9]'`)
cameranum=${#cameralist[@]}
eventsfolder=/usr/share/zoneminder/events/

echo number of detected zoneminder monitors: ${#cameralist[@]}

echo starting list generation on all cameras
for (( i=0; i<${cameranum}; i++ ));
	do
		echo starting list generation on zoneminder monitor: ${cameraname[$i]}
		find $eventsfolder${cameralist[$i]} -mtime -1 -name \*capture.jpg > /tmp/alarmvideos/${cameraname[$i]}.list
		echo finished list generation on zoneminder monitor: ${cameraname[$i]}

		echo starting list sorting on zoneminder monitor: ${cameraname[$i]}
		sort /tmp/alarmvideos/${cameraname[$i]}.list -v -o /tmp/alarmvideos/${cameraname[$i]}-sorted.list
		echo finished list sorting on zoneminder monitor: ${cameraname[$i]}
	done
echo finished list generation on all cameras

echo starting video generation on all cameras
for (( i=0; i<${cameranum}; i++ ));
	do
		echo starting video generation on zoneminder monitor: ${cameraname[$i]}
		mencoder mf://@/tmp/alarmvideos/${cameraname[$i]}-sorted.list -mf w=640:h=480:fps=15:type=jpg \
			-ovc lavc -lavcopts vbitrate=600:turbo:threads=4:vcodec=mpeg4:vmax_b_frames=0:vme=4:mbd=1:autoaspect:dia=1:vb_strategy=0:predia=1:last_pred=0 \
	#		-ovc xvid -xvidencopts bitrate=600:turbo:nochroma_me:notrellis:max_bframes=0:vhq=0:threads=4 \
	#		-ovc x264 -x264encopts bitrate=700:subq=4:bframes=2:b_pyramid=normal:weight_b:threads=auto \
			-oac copy \
			-o /tmp/alarmvideos/`date +%d`-${cameraname[$i]}.avi
		echo finished video generation on zoneminder monitor: ${cameraname[$i]}
	done
echo finished video generation on all cameras

# create directory for year and month (if it's not existing)
#mkdir -p /yourarchivedirectory/images/videos/`date +%y`/`date +%m`/

# find avi's with actual events-move to new dir
#cd /tmp/alarmvideos
#find *.avi -size +5k -exec mv {} /yourarchivedirectory/images/videos/`date +%y`/`date +%m` \;

# cleanup temp files
#rm -rf /tmp/alarmvideos/*.list
#rm -rf /tmp/alarmvideos/*.avi
	
# cleanup after 1 year archive 
#find /yourarchivedirectory/images/videos* -mtime +365 -exec rm {} \;
vraa
Posts: 31
Joined: Mon May 23, 2011 6:28 am

Re: Daily alarm events video script

Post by vraa »

This is my code now

I've added lots of small tricks and what not so you can either use it as a cron script or you can use it as a one time thing if you need a specific days video

Code: Select all

#!/bin/bash

# use this script for creating alarm video-files with zoneminder 1.25.0
# by nightcrawler
# edited by vraa (ubuntu friendly, zoneminder 1.24 friendly, h264 friendly, 1000+ frame friendly)

#make temp directory
mkdir -p /tmp/alarmvideos

# enumerate existing zoneminder monitors
cameraname=(`ls /usr/share/zoneminder/events/ | grep '[a-za-z]'`)
cameralist=(`ls /usr/share/zoneminder/events/ | grep '^[0-9]'`)
cameranum=${#cameralist[@]}
eventsfolder=/usr/share/zoneminder/events/
#date=(`date +%d-%m-%y`)
date=03-06-12

#date config
#year=(`date +%y`)
#month=(`date +%m`)
#day=(`date +%d`)
year=12
month=06
day=03
directorypath=/$year/$month/$day
#end date config

echo number of detected zoneminder monitors: ${#cameralist[@]} on $date

echo starting list generation on all cameras
for (( i=0; i<${cameranum}; i++ ));
	do
		echo starting list generation on zoneminder monitor: ${cameraname[$i]} at $eventsfolder${cameralist[$i]}$directorypath
#		find $eventsfolder${cameralist[$i]} -mtime -1 -name \*capture.jpg > /tmp/alarmvideos/${cameraname[$i]}.list
#		find -O3 $eventsfolder${cameralist[$i]}$directorypath -daystart -mtime 4 -name \*capture.jpg > /tmp/alarmvideos/$date-${cameraname[$i]}.list
		echo finished list generation

		echo starting list sorting
#		sort /tmp/alarmvideos/$date-${cameraname[$i]}.list -V -o /tmp/alarmvideos/$date-${cameraname[$i]}-sorted.list
		echo finished list sorting
	done
echo finished list generation on all cameras

#			-ovc xvid -xvidencopts bitrate=600:turbo:nochroma_me:notrellis:max_bframes=0:vhq=0:threads=4 \
#			-ovc x264 -x264encopts bitrate=700:subq=4:bframes=2:b_pyramid=normal:weight_b:threads=auto \

echo starting video generation on all cameras
for (( i=0; i<${cameranum}; i++ ));
	do
		echo starting video generation on zoneminder monitor: ${cameraname[$i]}
		mencoder mf://@/tmp/alarmvideos/$date-${cameraname[$i]}-sorted.list -mf w=640:h=480:fps=15:type=jpg \
			-ovc lavc -lavcopts vbitrate=600:turbo:threads=4:vcodec=mpeg4:vmax_b_frames=0:vme=4:mbd=1:autoaspect:dia=1:vb_strategy=0:predia=1:last_pred=0 \
			-oac copy \
			-o /tmp/alarmvideos/$date-${cameraname[$i]}.avi
		echo finished video generation
	done
echo finished video generation on all cameras

# create directory for year and month (if it's not existing)
#mkdir -p /yourarchivedirectory/images/videos/`date +%y`/`date +%m`/

# find avi's with actual events-move to new dir
#cd /tmp/alarmvideos
#find *.avi -size +5k -exec mv {} /yourarchivedirectory/images/videos/`date +%y`/`date +%m` \;

# cleanup temp files
#rm -rf /tmp/alarmvideos/*.list
#rm -rf /tmp/alarmvideos/*.avi

# cleanup after 1 year archive
#find /yourarchivedirectory/images/videos* -mtime +365 -exec rm {} \;
Post Reply