Replaying Events for Testing Zones / Using a video file as a Source

Add any particular hints or tricks you have found to help with your ZoneMinder experience.
Post Reply
User avatar
kitkat
Posts: 193
Joined: Sun Jan 27, 2019 5:17 pm

Replaying Events for Testing Zones / Using a video file as a Source

Post by kitkat »

There was a question a while ago asking whether it was possible to test zones against stored events, which interested me, and I I've cobbled together a fairly reliable method of doing it with PHP and FFMpeg.

Create a file named replay-test.php in your server's web root (not the ZM root) and paste this into it. You shouldn't have to change anything, just copy/paste/save.

Code: Select all

<?PHP

// Usage / ZM Source syntax
// To stream a ZM event: http://server/replay-test.php?e=eventID
// To stream a video file: http://server/replay-test.php?f=/full/path/to/file.mp4

// Source file
// Used only if you don't set an eventID below and don't
// pass a filename or event ID in the URL query string.
// Must be the /full/path/to/file.mp4
$infile = "";

// Event ID
// Used only if you don't pass an eventID or filename in the URL query string.
// Use 0 for none
$eventID = 0;

// Database info
// We need this when using event IDs.
// We'll try to get to get it from the
// ZM conf files so you'll only need to
// set it if that fails and you get an error.
// If you set one item then you must set all.
$db = array(    "ZM_DB_HOST" => "",
		"ZM_DB_NAME" => "",
		"ZM_DB_USER" => "",
		"ZM_DB_PASS" => "" );

// Whether to loop the video
// Use -1 for infinite,
// 0 for single-shot,
// 1 for 1 loop (2 plays), etc.
// Anything other than -1 will throw warnings in the ZM
// log when the stream ends and the zmc_ process restarts.
$loopcount = -1;

// Path to ffpmeg - try 'which ffmpeg' at the CLI to find yours
$ffmpeg = "/usr/bin/ffmpeg";

// Path to grep
$grep = "/bin/grep";

// No configuration variables below here

// URL parameters
if( !empty($_GET['f']) ) {
	$infile = $_GET['f'];
	$eventID = 0;
}

if( !empty($_GET['e']) ) {
	$eventID = $_GET['e'];
}

// Got anything to do?
if( empty($infile) && empty($eventID) ) {
	header("HTTP/1.1 400 Bad Request (no filename or event ID)");
	echo "No filename or event ID";
	exit;
}

// Get a filepath if we've got an eventID
if( !empty($eventID) ) {
	$infile = getFilenameByEventID($eventID);
}

// Can we read the file?
if( empty($infile) || !is_readable($infile) ) {
	header("HTTP/1.1 500 Internal Server Error (can't read file)");
	echo "Can't read file '".$infile."')";
	exit;
}

// ffmpeg good to go?
if( !is_executable($ffmpeg) ) {
	header("HTTP/1.1 500 Internal server error (can't execute ffmpeg)");
	echo "Can't execute ffmpeg - please set the path";
	exit;
}

// Don't let max_execution_time kill us
set_time_limit(0);

// No output buffering or compression
// Try to cover all the bases...
@apache_setenv('no-gzip', 1);
@ini_set('output_buffering', 'Off');
@ini_set('output_handler', '');
@ini_set('zlib.output_compression', 'Off');
@ob_end_flush();
@flush;

// Tell the client what we're sending
header( "Content-Type: video/mp4" );

// And invoke FFmpeg to send it...
passthru( $ffmpeg." -stream_loop ".$loopcount." -re -fflags +genpts -i '".$infile."' -vcodec copy -an -sn -vsync 0 -map_metadata -1 -f mp4 -movflags frag_keyframe+empty_moov - 2>/dev/null");

// All done

exit;

function getFilenameByEventID( $eventID ) {
	// We'll need these
	// Don't like globals but don't want to pass anything other than eventID
	$db = $GLOBALS['db'];
	$grep = $GLOBALS['grep'];

	// Do we need to get database details from the .conf?
	if( empty($db['ZM_DB_HOST']) ) {
		if( !is_executable($grep) ) {
			header("HTTP/1.1 500 Internal server error (can't execute grep)");
			echo "Can't execute grep - please set the path";
			exit;
		}
		exec("$grep -hE \"^\s*ZM_DB_\" /etc/zm/zm.conf /etc/zm/conf.d/*.conf", $db);
		foreach($db as $key => $value) {
			unset($db[$key]);
			$newkey = trim(preg_replace("/^\s*(ZM_DB_[^=]*)=\s*(.*)\s*$/", "$1", $value ));
			$newvalue = trim(preg_replace("/^\s*(ZM_DB_[^=]*)=\s*(.*)\s*$/", "$2", $value ));
			if($newkey) {
				$db[$newkey] = $newvalue;
			}
		}
		unset($value);
	}

	// Got 'em all?
	if( empty($db['ZM_DB_HOST']) || empty($db['ZM_DB_NAME']) || empty($db['ZM_DB_USER']) || empty($db['ZM_DB_PASS']) ) {
		header("HTTP/1.1 500 Internal server error (database info incomplete)");
		echo "Database info incomplete - Please set it manually";
		exit;
	}

	// Connect to the database
	mysqli_report(MYSQLI_REPORT_OFF);
	if( !$dbHandle = @mysqli_connect($db['ZM_DB_HOST'], $db['ZM_DB_USER'], $db['ZM_DB_PASS'], $db['ZM_DB_NAME']) ) {
		header("HTTP/1.1 500 Internal server error (can't connect to database)");
		echo "Can't connect to database: ".mysqli_error($dbhandle);
		exit;
	}

	// Query it for the file path
	$query = "SELECT CONCAT(s.`path`, '/', e.`MonitorId`, '/', LEFT(e.`StartDateTime`, 10), '/', e.`Id`, '/', e.`defaultVideo`)
		FROM `Events` e
		INNER JOIN `Storage` s
		ON s.`Id` = e.`StorageID`
		WHERE e.`id` = '".mysqli_real_escape_string($dbHandle, $eventID)."'";

	if( !$result = @mysqli_query($dbHandle, $query) ) {
		header("HTTP/1.1 500 Internal server error (database query failed)");
		echo "Database query failed: ".mysqli_error($dbHandle);
		exit;
	}

	// Got a result?
	if( (!$fullpath = @mysqli_fetch_row($result)) || empty($fullpath[0]) ) {
		header("HTTP/1.1 500 Internal server error (empty result set or path)");
		echo "Mysql returned empty result or path (bad event ID?)";
		exit;
	}

	mysqli_close($dbHandle);

	return $fullpath[0];
}

?>
You can test the stream by opening it in a browser or a media player such as MPC-HC or VLC.

If you entered a filename or an event ID in the code then you can simply visit http://yourserver/replay-test.php

If you didn't enter a filename or event ID in the code then you'll need to add one to the url.

If you're using an event ID then the URL format is yourserver/replay-test.php?e=eventID (substituting the actual event ID for 'eventID')

If you want to use a file then the format is yourserver/replay-test.php?f=/full/path/to/file.mp4

If the video doesn't play and you're using a media player then try a browser - there'll probably be something useful in the output.

If you're using a browser and it just shows a black screen then the video might be H.265, which most of them don't support.

Once you've verified that it's working you can set up a monitor in ZM with a Source Type of FFMpeg, a Source Path as described above, and TCP as the Method.

That's it - You should be good to go and ZM should start capturing :)

The video is passed straight through rather than being transcoded so playback quality will be exactly as it was recorded in the file.

The best results will probably be obtained with videos that used Camera Passthrough as the writer because then ZM will see the same thing as it did the first time - other methods will be lossy and although the differences may be imperceptible to us, machines will spot them. If you must use an encoder then try to use a low CRF, such as 19 or even 17 to avoid too much degradation.

If you change the source video path or event ID in the PHP or if you overwrite the video file then you'll have stop and restart the monitor in ZM.

I find that the loop tends to stick at the end before restarting, but that could very well be the way my video is encoded and it may not happen to you.

I suggest using sources with a few fairly static frames at their start end end to avoid spurious detection at the wraparound where the scene might change significantly (in ZM terms).

I think that's all - Good luck!
Last edited by kitkat on Fri Dec 03, 2021 3:16 pm, edited 2 times in total.
User avatar
burger
Posts: 434
Joined: Mon May 11, 2020 4:32 pm

Re: Replaying Events for Testing Zones / Using a video file as a Source

Post by burger »

This sounds like it would be useful combined with the script by LDBG that can make timelapse videos here: viewtopic.php?f=9&p=123671#p123671

You can make a video of say 24 hours, and then run detection on it.

Also a good way to fine tune zones, if you are so inclined.

Edit: Thinking on this again i thought maybe source type:file could be used for this. Well that doesn't work but ffmpeg source was able to watch a video file albeit the fps must be specified, otherwise it plays back at some hilarious 1000fps. So ffmpeg can be used. To avoid permissions issues i put the file in tmp and assigned it to www-data (debian).

I forget if live audio playback is enabled in 1.36 or 1.37 yet, but if so it then makes ZM able to stream videos as a server which is now getting out of security territory. Not sure whether that's good or bad but it works. Note that the fps on ZM measures 31 while the video src file was 29.970. There might be some slight time difference.
Attachments
screenshot.png
screenshot.png (60.83 KiB) Viewed 91779 times
fastest way to test streams:
ffmpeg -i rtsp://<user>:<pass>@<ipaddress>:554/path ./output.mp4 (if terminal only)
ffplay rtsp://<user>:<pass>@<ipaddress>:554/path (gui)
find paths on ispydb or in zm hcl

If you are new to security software, read:
https://wiki.zoneminder.com/Dummies_Guide
User avatar
kitkat
Posts: 193
Joined: Sun Jan 27, 2019 5:17 pm

Re: Replaying Events for Testing Zones / Using a video file as a Source

Post by kitkat »

I've updated the code in the first post and the Wiki article to work with event IDs, which is a lot easier than faffing around with file paths. You can still use file paths if you want to though.
Last edited by kitkat on Fri Dec 03, 2021 5:29 pm, edited 1 time in total.
User avatar
kitkat
Posts: 193
Joined: Sun Jan 27, 2019 5:17 pm

Re: Replaying Events for Testing Zones / Using a video file as a Source

Post by kitkat »

burger wrote: Wed Dec 01, 2021 2:48 am Edit: Thinking on this again i thought maybe source type:file could be used for this. Well that doesn't work but ffmpeg source was able to watch a video file albeit the fps must be specified, otherwise it plays back at some hilarious 1000fps.
Yeah, @haus tried that in the other thread but I think ZM choked on the framerates.

[e2a: And when I tried it I thought it hadn't worked because it was all over so quickly that the Console didn't have a chance to show it was capturing, and the log filled up with "Failed to capture..." messages. The solution seemed to be to use the -re option before the input file to get FFmpeg to read at real-time speed, but there didn't seem to be way to do that in ZM however many ways I tried.]

I knocked up a Bash script that created JPEG files from the video frames with each one overwriting the previous one at about 1 FPS and that worked with a Source Type of Remote but there's a quality loss doing it that way and although I tried PNG instead of JPEG, ZM wouldn't recognise them.

Framerate is also a bit hit and miss that way - It's difficult to get an actual framerate when using mv/cp and one can really only set a delay or interval between each action so there's a sync issue over time, which is compounded by the ZM sampling intervals.


e2a: I've found that the framerate shown in ZM, especially in the Monitor view, is generally a bit different to the actual rate, but I don't doubt that this script may well introduce timing errors of its own.
haus
Posts: 213
Joined: Thu Oct 11, 2007 5:10 am

Re: Replaying Events for Testing Zones / Using a video file as a Source

Post by haus »

This is fantastic and working nicely! Really appreciate the effort you put into it.
haus
Posts: 213
Joined: Thu Oct 11, 2007 5:10 am

Re: Replaying Events for Testing Zones / Using a video file as a Source

Post by haus »

I removed this post. I had to open more ports in my router and apache config, sorry about that. Testing this later!
haus
Posts: 213
Joined: Thu Oct 11, 2007 5:10 am

Re: Replaying Events for Testing Zones / Using a video file as a Source

Post by haus »

OK, I finally got this working as a monitor, and I copied the zone from the original monitor exactly, but it's not creating alarms. I suspect the reason is the frame rate on the video file monitor is much lower than the actual camera.

On the original camera I get State: Idle - 4fps 3.63 fps 3.63 fps

On the replay-test monitor I get State: Idle - 0 fps 3.58fps 3.44 fps (sometimes 1fps)

The frame rate on the replay-test monitor is visibly slow and seems to freeze up a lot. I don't think it's a server load issue because that hasn't changed and I'm able to view the original camera at its natural frame rate in another window. Maybe the fps of the original camera is too low, or I need to change a setting for ffmpeg?
dougmccrary
Posts: 1322
Joined: Sat Aug 31, 2019 7:35 am
Location: San Diego

Re: Replaying Events for Testing Zones / Using a video file as a Source

Post by dougmccrary »

Just a stab - did the original camera have any General -> * FPS fields set?
haus
Posts: 213
Joined: Thu Oct 11, 2007 5:10 am

Re: Replaying Events for Testing Zones / Using a video file as a Source

Post by haus »

No, they're all blank. The frame rate is set to 4 in the camera though. Maybe it's too low for this to work? I can try temporarily bumping it up today and creating an event to see what happens.
Detlef Paschke
Posts: 27
Joined: Tue Sep 12, 2023 6:14 pm

Re: Replaying Events for Testing Zones / Using a video file as a Source

Post by Detlef Paschke »

Hello,

I think the brackets were missing in line 91.
@flush();
This is how video files work for me.

In VSCode I get another error in another section.

Code: Select all

// Connect to the database
mysqli_report(MYSQLI_REPORT_OFF);
if( !$dbHandle = @mysqli_connect($db['ZM_DB_HOST'], $db['ZM_DB_USER'], $db['ZM_DB_PASS'], $db['ZM_DB_NAME']) ) {
	header("HTTP/1.1 500 Internal server error (can't connect to database)");
	echo "Can't connect to database: ".mysqli_error($dbHandle);
	exit;
}
Could there be an error in this area? I can not find it. VSCode says, "Expected type 'mysqli'. Found 'false'.".

Calling with the EventID doesn't work for me.

If I enter my data from line 24 under "$db = array(", the browser shows a 500 error.

"[Sun Oct 15 12:40:17.861024 2023] [php:error] [pid 81729] [client 192.168.0.2:51524] PHP Fatal error: Uncaught TypeError: mysqli_error(): Argument #1 ($mysql) must be of type mysqli, bool given in /var/www/html/replay-test.php:138\nStack trace:\n#0 /var/www/html/replay-test.php(138): mysqli_error()\n# 1 /var/www/html/replay-test.php(63): getFilenameByEventID()\n#2 {main}\n thrown in /var/www/html/replay-test.php on line 138"

If I don't specify anything, I get the message "Mysql returned empty result or path (bad event ID?)".

Best regards
Detlef Paschke
Post Reply