Record High-res H264 streams 24/7 with low CPU Load

[bbarnett]

Make that 4 patches.

This one is in the original code.

Line 283 from original is this:

if [ $ELAPSED_SECONDS > 0 ]

correct is this:

if [ $ELAPSED_SECONDS -gt 0 ]

I've patched it in my version, and just re-uploaded the whole file.

[xefil]

Hello,

Here the code after your patches (hope I've applied them correctly) and the SQL correction previously mentioned:

Code: Select all

#!/bin/bash

# Original script posted here https://forums.zoneminder.com/viewtopic.php?f=9&t=27537  by russell_i_brown 

# Install to /opt/zmrecord.sh

#       Grab a video stream from an IP camera into a temp file.
#       Inject an 'Events' record into the ZM Database.
#       Create the directory in the zm storage area
#       Move the mp4 to the right place
#
#       Call with the Camera Id number as arg1:
#
#       zmrecord.sh 12
#
#
#       You might have to install mediainfo which I use to grab
#       the frame count.
#
#       Version 0.1     -  russell_i_brown      18/ 9/18        First hack
#       Version 0.2     -  russell_i_brown      19/ 9/18        Read monitor guff from DB
#       Version 0.3     -  russell_i_brown      19/ 9/18        Set StorageId in DB
#       Version 0.4     -  matt_watts           10/12/19        Use FFMPEG segment output to ensure frames are not missed between segments
#       Version 0.5     -  matt_watts           12/12/19        Move video segments immediatley into database / storage tree while live and keep db stats updated
#       Version 0.6     -  matt_watts           16/12/19        - Replace FFMPEG list output with simple folder polling allowing some crash/script kill recovery
#                                                               - Use FFMPEG empty_moov flag to allow playing in-recording segment in the web ui and preventing
#                                                                 corrupt video on crash/script kill
#                                                               - Continuous event database updates as segment is being wrote
#                                                               - Log some errors and events direct to Zoneminders DB Log
#       Version 0.7     -  russell_i_brown      31/01/20        Use Event Prefix as requested by tuxmos
#                                                               Handle Monitor with a StorageId of 0 by defaulting to 1 if blank  (ZM 1.34 issue?)
#                                                               Calculate the Bandwidth so the ZM console looks sane.

#########################################
#       Recording chunk time in seconds
#       This script will record for this number of seconds at a time
#       600 (10 minutes) is what I use.
RECORD_TIME=600

# MySQL
# set the user this way, if needed
# (NOTE: you could pass other options here, if needed, such as -h, etc)
#MYSQL_AUTH=()
MYSQL_AUTH=(-h <MySQLHost> -u zmuser)

# set the password here, if needed
export MYSQL_PWD="SECRET_PWD"

# some checks, as this script depends upon these things
# (leaving stderr unvarnished, so end user gets more info)

if ! mediainfo --version >/dev/null; then
    echo "Problem with mediainfo, maybe apt-get install mediainfo?"
    exit 1
fi

if ! bc -v >/dev/null; then
    echo "Problem with bc, maybe apt-get install bc?"
    exit 1
fi

# 0 or 1
debug=1

#
#########################################

trap "kill 0" SIGINT

# ZM Log
function zm_log() {
        CODE="$1"
        CODEVAL="$2"
        MESSAGE="`echo $3|tr -d '"'`"
        TIMESTAMP=`date +%s`
        SQL="INSERT INTO Logs ( Component,File,ServerId,Pid,Level,Code,Line,TimeKey, Message ) 
             VALUES           ( 'zmrecord.sh', 'zmrecord.sh', NULL, $$, $CODEVAL, \"$CODE\", 0, $TIMESTAMP, \"$MESSAGE\" );"
        echo $SQL | mysql ${MYSQL_AUTH[@]} -NBqr zm
}


function log_inf() {
        zm_log "INF" "2" "$@"
}

function log_err() {
        zm_log "ERR" "-2" "$@"
}

# Run the ffmpeg process and keep stats file up to date while it runs - pass ffmpeg errors to zm log
function run_ffmpeg() {
        MONITOR_ID="$1"
        CAM_URL="$2"
        OUTPUT_STATS_FN="$3"
        RECTIME="$4"
        OUTPUT_FN="$5"

        rm "$OUTPUT_STATS_FN".errorpipe 2>/dev/null
        mkfifo "$OUTPUT_STATS_FN".errorpipe
        rm "$OUTPUT_STATS_FN".statpipe 2>/dev/null
        mkfifo "$OUTPUT_STATS_FN".statpipe

        cat "$OUTPUT_STATS_FN".errorpipe | \
        while read ERR; do
                log_err "Monitor: $MONITOR_ID FFMPEG: $ERR"
        done &
        PID_ERR_READ=$!

        cat "$OUTPUT_STATS_FN".statpipe | \
        while read LN; do
                PARM="`echo $LN | cut -d '=' -f1`"
                VAL="`echo $LN | cut -d '=' -f2`"
                if [ "$PARM" = "fps" ]; then FFMPEG_FPS=$VAL; fi
                if [ "$PARM" = "bitrate" ]; then FFMPEG_BITRATE=$VAL; fi
                if [ "$PARM" = "frame" ]; then FFMPEG_FRAME=$VAL; fi
                echo "`date +%s` $FFMPEG_FPS $FFMPEG_BITRATE $FFMPEG_FRAME" > "$OUTPUT_STATS_FN"
        done &
        PID_STAT_READ=$!

        ffmpeg -y -loglevel error -i "$CAM_URL" -c copy ${DOAUDIO[@]} -f segment \
               -segment_time $RECTIME -segment_atclocktime 1 -segment_format_options movflags=empty_moov -strftime 1 "$OUTPUT_FN" \
               -progress - -nostats 2>"$OUTPUT_STATS_FN".errorpipe 1>"$OUTPUT_STATS_FN".statpipe &
        PID_FFMPEG=$!

        # Wait for exit signal or child pid death
        RUNNING=1
        trap 'RUNNING=0' EXIT
        trap 'RUNNING=0' INT
        trap 'RUNNING=0' TERM
        while [ "$RUNNING" = "1" ]; do
                if ! kill -0 "$PID_FFMPEG" 2>/dev/null; then RUNNING=0; fi
                if ! kill -0 "$PID_ERR_READ" 2>/dev/null; then RUNNING=0; fi
                if ! kill -0 "$PID_STAT_READ" 2>/dev/null; then RUNNING=0; fi
                sleep 1
        done
        # Shutdown the subprocess - we want to make really really sure that FFMPEG has indeed died before leaving!
        # its not 100% perfect and the output from netcams can really freeze it occassionaly to the point SIGINT
        # does not work and the subprocess exiting will leave a zombie ffmpeg behind
        # if it does not shutdown in 4 seconds force kill it
        kill $PID_FFMPEG $PID_ERR_READ $PID_STAT_READ 2>/dev/null
        KILL_TIMEOUT=0
        RUNNING=1
        trap 'RUNNING=1' EXIT
        trap 'RUNNING=1' INT
        trap 'RUNNING=1' TERM
        while [ "$RUNNING" = "1" ]; do
                KILL_TIMEOUT=$((KILL_TIMEOUT+1))
                if ! kill -0 "$PID_FFMPEG" 2>/dev/null; then RUNNING=0; fi
                if [ "$KILL_TIMEOUT" -gt 4 ]; then
                        kill -9 $PID_FFMPEG 2>/dev/null
                fi
                sleep 1
        done
        rm "$OUTPUT_STATS_FN".errorpipe 2>/dev/null
        rm "$OUTPUT_STATS_FN".statpipe 2>/dev/null

        log_err Monitor: $MONITOR_ID FFMPEG Has Terminated
}

# Scan the tmp folder for <timestamp>.mp4 files
# Move them into the correct tree and create a DB entry ensuring that on premature script termination what data was
# captured so far is in place
# leave an eventid and symbolic link in the tmp folder 
# the eventid keeps the info for this script to poll the video in its new location
# the symbolic link allows ffmpeg to re-open the file. FFMPEG does this when quit earlier than expected to add the moov atom (faststart)
function addIncompleteVideosToDB() {
        TMPFOLDER="$1"
        MONITOR_ID="$2"
        STORAGEID="$3"
        RECORD_TIME="$4"
        WIDTH="$5"
        HEIGHT="$6"
        STORE_PATH="$7"
        EVENT_PREFIX="$8"
        # iterate all new mp4 files found
        for FILEPATH in "$TMPFOLDER"/*.mp4; do
                [ -f "$FILEPATH" ] || continue
                # get beginning timestamp from ffmpeg created filename
                FILENAME=${FILEPATH##*/}
                TIMESTAMP=${FILENAME%%.mp4}
                # create new db entry for video segment
                SQL="   INSERT INTO Events 
                        (       MonitorId,StorageId,Name,Cause,StartDateTime,EndDateTime,Width,Height,Length,Frames,
                                Videoed,DiskSpace,Scheme,StateId,AlarmFrames,SaveJPEGs,Notes)
                        VALUES
                        (       $MONITOR_ID,\"$STORAGEID\",\"New Event\",\"Continuous\",FROM_UNIXTIME($TIMESTAMP),
                                NULL,$WIDTH,$HEIGHT,1,1,1,1,\"Medium\",1,0,0,\"manual ffmpeg\");
                        SET @last_id = LAST_INSERT_ID();
                        INSERT into Frames set EventId=@last_id,FrameId=1,Type=\"Normal\",TimeStamp=FROM_UNIXTIME($TIMESTAMP),Delta=0.00,Score=0;
                        UPDATE Events SET DefaultVideo=CONCAT(@last_id,\"-video.mp4\") where Id=@last_id;
                        SELECT @last_id;"
                THIS_ID=`echo $SQL | mysql ${MYSQL_AUTH[@]} -NBqr zm`
                # if mysql returned a new eventid then move the video into the correct place
                # ffmpeg will still retain its handle to the file and continue writing
                # leave an eventid reference to the video in the tmp folder - this will allow 
                # polling of the file until it needs completing 
                if ! [ "$THIS_ID" -le 0 ]; then
                        DATE_PATH=$STORE_PATH/`date -d @$TIMESTAMP --rfc-3339=date`
                        THIS_PATH=$DATE_PATH/$THIS_ID
                        NEW_VIDEO_PATH="$THIS_PATH"/$THIS_ID-video.mp4
                        DESCFILE="$TMPFOLDER"/"$TIMESTAMP".eventid
                        mkdir -p $THIS_PATH
                        mv -f "$FILEPATH" "$NEW_VIDEO_PATH"
                        chown www-data:www-data "$NEW_VIDEO_PATH" "$THIS_PATH" "$DATE_PATH"
                        echo "$THIS_PATH/$THIS_ID-video.mp4" > "$DESCFILE"
                        log_inf "Monitor: $MONITOR_ID - New segment : $THIS_PATH/$THIS_ID-video.mp4 Event $THIS_ID"
                fi
        done
}

# Scan the tmp folder for <timestamp>.eventid files
# By checking which ones point to an mp4 that is not open ( by ffmpeg ) we can determine
# the segment is complete and close it. This can also pick up stray recordings after say a PC crash
function addCompleteVideosToDB() {
        TMPFOLDER="$1"
        MONITOR_ID="$2"
        STORAGEID="$3"
        RECORD_TIME="$4"
        WIDTH="$5"
        HEIGHT="$6"
        STORE_PATH="$7"
        # iterate all eventid files in the tmp folder
        for EVENTID_FILEPATH in "$TMPFOLDER"/*.eventid; do
                [ -f "$EVENTID_FILEPATH" ] || continue
                EVENTID_FILENAME=${EVENTID_FILEPATH##*/}
                TIMESTAMP=${EVENTID_FILENAME%%.eventid}
                [[ $TIMESTAMP =~ ^[0-9]+$ ]] || continue
                VIDEO_PATH="`cat "$EVENTID_FILEPATH"`"
                VIDEO_FILENAME=${VIDEO_PATH##*/}
                EVENTID=${VIDEO_FILENAME%%-video.mp4}

                # if the video is not longer open for writing remove the eventid file, this will be the last update
                LASTUPDATE=0
                if [ "`fuser "$VIDEO_PATH" 2>/dev/null`" = "" ]; then
                        log_inf "Monitor: $MONITOR_ID Event : $EVENTID - Segment complete : $VIDEO_PATH"
                        rm "$EVENTID_FILEPATH"
                        LASTUPDATE=1
            # as this is the last run, change the name
            SQL="UPDATE Events SET Name=\"${EVENT_PREFIX}${EVENTID}\" where Id=$EVENTID;"
            echo $SQL | mysql ${MYSQL_AUTH[@]} -NBqr zm
                fi
                # get video stats and validate them
                # only spit out actual errors if this is the final update (video finished), an in-writing video
                # is likley to not be stat ready early in its life
                FRAMES=`mediainfo --Output="Video;%FrameCount%" "$VIDEO_PATH"`
                if ! [[ $FRAMES =~ ^[0-9]+$ ]]; then
                        if [ $LASTUPDATE -eq 1 ]; then
                                log_err "Monitor: $MONITOR_ID Event : $EVENTID ERROR: mediainfo --Output=Video;%FrameCount% $VIDEO_PATH returned '$FRAMES', probable corrupt video"
                        fi
                        FRAMES=1
                fi
                FILESIZE=`stat -c%s "$VIDEO_PATH"`
                if ! [[ $FILESIZE =~ ^[0-9]+$ ]]; then
                        if [ $LASTUPDATE -eq 1 ]; then
                                log_err "Monitor: $MONITOR_ID Event : $EVENTID ERROR: stat -c%s $VIDEO_PATH returned '$FILESIZE', integer expecected"
                        fi
                        FILESIZE=1
                fi
                END_TIMESTAMP=`stat -c %Y "$VIDEO_PATH"`
                if ! [[ $END_TIMESTAMP =~ ^[0-9]+$ ]]; then
                        if [ $LASTUPDATE -eq 1 ]; then
                                log_err "Monitor: $MONITOR_ID Event : $EVENTID ERROR: stat -c %Y $VIDEO_PATH returned '$END_TIMESTAMP', integer expecected"
                        fi
                        LENGTH=$RECORD_TIME
                        END_TIMESTAMP=$((TIMESTAMP+1))
                fi
                # update the db with stats
                SQL="   UPDATE Events SET 
                                Frames=$FRAMES,
                                EndDateTime=FROM_UNIXTIME($END_TIMESTAMP),
                                DiskSpace=$FILESIZE
                                WHERE Id=$EVENTID;
                        UPDATE Events SET Length=UNIX_TIMESTAMP(EndDateTime)-UNIX_TIMESTAMP(StartDateTime) 
                        WHERE StartDateTime IS NOT NULL AND EndDateTime IS NOT NULL AND Id=$EVENTID;"
                echo $SQL | mysql ${MYSQL_AUTH[@]} -NBqr zm
        if [ $LASTUPDATE -eq 1 ]; then
            if [ $FRAMES -gt 1 ]; then
                    LENGTH=$(echo "scale=0;$(mediainfo --Output="Video;%Duration%" "$VIDEO_PATH") / 1000" | bc)
                    STARTTIME=$((END_TIMESTAMP - $LENGTH))
                # if we see a linked_monitor_id, then try to detect alarm frames there
                if [[ $LINKED_MONITOR_ID =~ ^[0-9]+$ ]]; then
                    SQL="SELECT count(*) from Events where MonitorId=$LINKED_MONITOR_ID and AlarmFrames > 0 and StartDateTime > FROM_UNIXTIME($STARTTIME) and EndDateTime < FROM_UNIXTIME($END_TIMESTAMP);"
                    #
                    ALARMFRAMES=$(echo $SQL | mysql ${MYSQL_AUTH[@]} -NBqr zm)
                    if [[ $ALARMFRAMES =~ ^[0-9]+$ ]] && [ $ALARMFRAMES -gt 0 ]; then
                        SQL="UPDATE Events set AlarmFrames=$ALARMFRAMES where id=$EVENTID"
                        echo "FOUND ALARM: $SQL"
                        echo $SQL | mysql ${MYSQL_AUTH[@]} -NBqr zm
                    fi
                fi
                #
                # calc number of bulk frames, and single frames
                BULKNUMBER=100
                FRAMERATE=$(mediainfo --Output="Video;%FrameRate%" "$VIDEO_PATH")
                if ! [[ $FRAMERATE =~ ^[0-9.]+$ ]]; then
                    log_err "Monitor: $MONITOR_ID Event : $EVENTID ERROR: mediainfo FrameRate check on "$VIDEO_PATH" returned '$FRAMERATE', integer expecected"
                    return
                fi
                
                skew=$(echo "scale=2; 1 / ${FRAMERATE}" | bc)
                skewbulk=$(echo "scale=2; $BULKNUMBER / ${FRAMERATE}" | bc)
                # we could leave out the 00 + bulk divide, but it may not be 100!
                COUNTINGIT=$(echo "scale=2; ${FRAMES}/${BULKNUMBER}" | bc)
                bulkcount=${COUNTINGIT:0:-3}
                # bc hates 0 pre-decimal
                if [ "$bulkcount" == "" ]; then
                    bulkcount=0
                fi
                # get remainder, plus force base 10 to strip and leading 0s.
                # (prevents interpretation as octal, etc)
                framecount=$((10#${COUNTINGIT: -2}))
                if [ $debug -eq 1 ]; then
                    echo "COUNTINGIT=$COUNTINGIT bc=$bulkcount fc=$framecount FRAMES=$FRAMES FRAMERATE=$FRAMERATE skew=$skew skewbulk=$skewbulk"
                fi
                # these need to be sensible numbers
                if [[ $bulkcount =~ ^[0-9]+$ ]] && [[ $framecount =~ ^[0-9]+$ ]] && [[ $skew =~ ^\.[0-9]+$ ]] && [[ $skewbulk =~ ^[0-9]+\.[0-9]+$ ]] && [[ $EVENTID =~ ^[0-9]+$ ]] && [[ $STARTTIME =~ ^[0-9]+$ ]]; then
                    
                    # clear out temp single frame
                    SQL="delete from Frames where EventId='$EVENTID';"
                    if [ $bulkcount -gt 0 ]; then
                        for (( i=0;i<=$bulkcount;i++)); do
                            currentskew=$(echo "scale=2; $skewbulk * $i" | bc)
                            if [ $i -gt 0 ]; then
                                currentbulk=$(echo "scale=0; ${BULKNUMBER} * $i" | bc)
                            else
                                currentbulk=1
                            fi
                            frametime=$(echo "scale=0; ${STARTTIME} + ${currentskew} / 1" | bc)
                            SQL+=$'\n'"INSERT into Frames (EventId,FrameId,Type,TimeStamp,Delta) values ('$EVENTID','$currentbulk','Bulk',FROM_UNIXTIME($frametime),'$currentskew');"
                        done
                    fi
                    # get current skew/bulk as a base
                    existingskew=$(echo "scale=2; $skewbulk * $bulkcount" | bc)
                    existingbulk=$(echo "scale=0; ${BULKNUMBER} * $bulkcount" | bc)
                    for (( i=1;i<=$framecount;i++)); do
                        currentskew=$(echo "scale=2; $skew * $i + $existingskew" | bc)
                        frametime=$(echo "scale=0; ${STARTTIME} + ${currentskew} / 1" | bc)
                        currentframe=$(echo "scale=0; $i + $existingbulk " | bc)
                        SQL+=$'\n'"INSERT into Frames (EventId,FrameId,Type,TimeStamp,Delta) values ('$EVENTID','$currentframe','Normal',FROM_UNIXTIME($frametime),'$currentskew');"
                    done

                    if [ $debug -eq 1 ]; then
                        echo "Writing SQL for EventId=$EVENTID"
                        echo "$SQL"
                    fi
                    #echo "$SQL"
                    echo $SQL | mysql ${MYSQL_AUTH[@]} -NBqr zm

                else
                    log_err "Monitor: $MONITOR_ID Event : $EVENTID ERROR: Value(s) bulkcount=$bulkcount framecount=$framecount skew=$skew skewbulk=$skewbulk not valid.  Cannot update Frame table."
                fi
            else
                log_err "Monitor: $MONITOR_ID Event : $EVENTID ERROR: Frame rate too low, unable to accurate add frame data."
            fi            
        fi
        done
}

# For Bandwidth Calc
LAST_FPS_TIME=0
LAST_FILESIZE=0
LAST_BW=0

# Keep the monitor status up to date so fps etc can be viewed in the zm console
function updateMonitorStatus() {
        MONITOR_ID="$1"
        STATS_FILE="$2"
        SHUTTINGDOWN="$3"
        STATUS="NotRunning"
        FPS=0
        BW=0
        if ! [ "$SHUTTINGDOWN" = "1" ]; then
                if [ -f "$STATS_FILE" ]; then
                        IFS=' ' read -r TS FPS BW FRAME <<< `cat "$STATS_FILE"`
            if [ $debug -eq 1 ]; then
                echo "TS:$TS FPS:$FPS BW:$BW FRAME:$FRAME"
            fi
                        if  [ -z "${TS//[0-9]}" ] && [ -n "$TS" ]; then
                                AGE=$(($(date +%s)-TS))
                        else
                                AGE=1000
                        fi
                        if [ "$AGE" -lt 4 ]; then
                                STATUS="Connected"
                        fi
                fi
        fi
        # Calc CaptureBandwith in the same way as ZM in zm_monitor.cpp:
        #       unsigned int new_capture_bandwidth = (new_camera_bytes - last_camera_bytes)/(now-last_fps_time);
        # Unfortunately, BASH can't trap a divide by zero so we have to check
        # Get Filesize for Bandwidth calc.  Ffmpeg buffers quite a lot so see if the file has changed
        # size before calculating the Bandwidth.
        FILESIZE=`stat -c%s "$NEW_VIDEO_PATH" 2>/dev/null`
    #
    # sometimes, the above read won't work.  The file is too small/just started
    # capturing, and therefore, nothing is read:
    # ./zmrecord.sh: line 283: 1615896773 - : syntax error: operand expected (error token is "- ")
    # so.. added a check here.  Regex means "exists" and "numeric" tests plus >0 are all in one
    #
        if [ ${FILESIZE:-0} -gt ${LAST_FILESIZE:-0} ] && [[ $TS =~ [0-9]+ ]]
        then
                ELAPSED_SECONDS=$(($TS - $LAST_FPS_TIME))
                if [ $ELAPSED_SECONDS > 0 ]
                then
                        FILESIZE_INCREASE=$(( ${FILESIZE:-0} - ${LAST_FILESIZE:-0} ))
                        LAST_BW=$(( $FILESIZE_INCREASE / $ELAPSED_SECONDS ))
                        LAST_FPS_TIME=$TS
                        LAST_FILESIZE=$FILESIZE
                fi
        fi
        SQL="   REPLACE INTO Monitor_Status (MonitorId, Status, CaptureFPS, CaptureBandwidth) 
                VALUES 
                ('$MONITOR_ID','$STATUS', '$FPS', ${LAST_BW:-0})"
        echo $SQL | mysql ${MYSQL_AUTH[@]} -NBqr zm
        echo $SQL
}

# Main function - 
function zmRecord() {
        MONITOR_ID=$1

        # Read Monitor Info From DB
        IFS=$'\t'
        read -r WIDTH HEIGHT READPATH STORAGEID EVENT_PREFIX <<< `mysql ${MYSQL_AUTH[@]} -NBqr zm -e "select Width,Height,Path,StorageId,EventPrefix from Monitors where Id=$MONITOR_ID"`
        read -r STORAGE <<< `mysql ${MYSQL_AUTH[@]} -NBqr zm -e "select Path from Storage where Id=$STORAGEID"`
        # Hmmmm.... on a fresh install of ZM 1.34, the Monitor has a StorageId of 0 but the Storage table
        # starts at Id 1.  Handle a blank Storage Path.
        if [ -z "$STORAGE" -a $STORAGEID -eq 0 ]
        then
                read -r STORAGE <<< `mysql ${MYSQL_AUTH[@]} -NBqr zm -e "select Path from Storage where Id=1"`
        fi
        STORE_PATH=$STORAGE/$MONITOR_ID
        if [ -z "$READPATH" -o -z "$STORE_PATH" -o -z "$STORAGE" ]; then
                log_err "$0 Monitor Data Error. Got: Width $WIDTH Height $HEIGHT ReadPath $READPATH StorePath $STORE_PATH"
                exit 1
        fi

        # create the working folders & files
        # stats txt is constantly updated and placed in ram ( /dev/shm )
        TMPFOLDER="$STORE_PATH/../$MONITOR_ID-zmrecordtmp"
        FFMPEG_STAT_OUTPUT_FILE=/dev/shm/zmrecord-m$MONITOR_ID-stats.txt
        FFMPEG_OUTPUT_FILENAME="$TMPFOLDER/%s.mp4"
        mkdir -p "$TMPFOLDER"
        rm "$FFMPEG_STAT_OUTPUT_FILE" 2>/dev/null
        touch "$FFMPEG_STAT_OUTPUT_FILE"
        chmod a+r "$FFMPEG_STAT_OUTPUT_FILE"

        # Main loop exit trap triggered by EXIT / INT / TERM signals
        RUNNING=1
        trap 'RUNNING=0' EXIT
        trap 'RUNNING=0' INT
        trap 'RUNNING=0' TERM
        # Main loop - start the ffmpeg processes and monitor
        # regularly run updates to check ffmpegs video and stat output and transfer
        # to zm database / file tree
        log_inf "Monitor: $MONITOR_ID Starting zmrecord.sh"
        FFMPEG_PID=-100
        while [ "$RUNNING" = "1" ]; do
                # testing this script one time my tmp folder vanished ( zm cleanup? ) 
                # and this script was unable to recover so I added this
                mkdir -p "$TMPFOLDER"
                # start subprocesses / restart them if they stopped
                if ! kill -0 "$FFMPEG_PID" 2>/dev/null; then
                        log_inf "Monitor: $MONITOR_ID Starting FFMPEG process"
                        run_ffmpeg "$MONITOR_ID" "$READPATH" "$FFMPEG_STAT_OUTPUT_FILE" "$RECORD_TIME" "$FFMPEG_OUTPUT_FILENAME" &
                        FFMPEG_PID=$!
                fi
                # check ffmpeg is actually working by monitoring the frame count
                FFMPEG_STATS_FRAMES=`cat "$FFMPEG_STAT_OUTPUT_FILE" | cut -d " " -f4`
                if [ "$PREV_FFMPEG_STATS_FRAMES" = "$FFMPEG_STATS_FRAMES" ]; then
                        if [ "$PREV_FFMPEG_STATS_FRAMES_SAMECOUNT" = "" ]; then
                                PREV_FFMPEG_STATS_FRAMES_SAMECOUNT=0
                        fi
                        PREV_FFMPEG_STATS_FRAMES_SAMECOUNT=$((PREV_FFMPEG_STATS_FRAMES_SAMECOUNT+1))
                else
                        PREV_FFMPEG_STATS_FRAMES_SAMECOUNT=0
                fi
                PREV_FFMPEG_STATS_FRAMES="$FFMPEG_STATS_FRAMES"
                if [ "$PREV_FFMPEG_STATS_FRAMES_SAMECOUNT" -gt 5 ]; then
                        if kill -0 "$FFMPEG_PID" 2>/dev/null; then
                                log_err "Monitor: $MONITOR_ID FFMPEG has stalled - killing"
                                kill $FFMPEG_PID
                                PREV_FFMPEG_STATS_FRAMES_SAMECOUNT=0
                        fi
                fi

                # run the video / db update polling functions
                addCompleteVideosToDB "$TMPFOLDER" "$MONITOR_ID" "$STORAGEID" "$RECORD_TIME" "$WIDTH" "$HEIGHT" "$STORE_PATH" "$EVENT_PREFIX"
                sleep 4
                updateMonitorStatus "$MONITOR_ID" "$FFMPEG_STAT_OUTPUT_FILE" 0
                addIncompleteVideosToDB "$TMPFOLDER" "$MONITOR_ID" "$STORAGEID" "$RECORD_TIME" "$WIDTH" "$HEIGHT" "$STORE_PATH" "$EVENT_PREFIX"
        done
        # Finish up
        if ! kill -0 "$FFMPEG_PID" 2>/dev/null; then
                kill "$FFMPEG_PID" 2>/dev/null
        fi
        addCompleteVideosToDB "$TMPFOLDER" "$MONITOR_ID" "$STORAGEID" "$RECORD_TIME" "$WIDTH" "$HEIGHT" "$STORE_PATH" "$EVENT_PREFIX"
        updateMonitorStatus "$MONITOR_ID" "$FFMPEG_STAT_OUTPUT_FILE" 1
        log_inf "Monitor: $MONITOR_ID Ended zmrecord.sh"
}

if [ $# -lt 1 ] || [[ ! $1 =~ ^[0-9]+$ ]] || [[ ! $2 =~ ^[01]$ ]] || [[ ! $3 =~ ^(|[0-9]+)$ ]]; then
        echo "Syntax: $0 <Monitor Id> <0|1>(record audio or not) <Linked Monitor Id>"
        echo "Script will search Linked Monitor for alarm frames, and add count to this monitor."
        exit 1
fi

MONITOR_ID=$1

# record audio toggle
if [ $2 -eq 1 ]; then
    DOAUDIO=(-c:a aac)
else
    DOAUDIO=()
fi

# if provided, verify monitor id exists
if [ "$3" != "" ] && [ $3 -gt 0 ]; then
    SQL="SELECT function from Monitors where id=$3;"
    THIS_ID=`echo $SQL | mysql ${MYSQL_AUTH[@]} -NBqr zm`
    if [ "$THIS_ID" == "" ]; then
        echo "Can't find linked monitor ID ${3}."
        exit 1
    fi
    if ! [[ $THIS_ID =~ (Modect|Mocord) ]]; then
        echo "It doesn't seem like this Linked Monitor is doing any motion detection.  Are you sure you want to use it?"
        echo "(Continuing anyway...)"
    fi
    LINKED_MONITOR_ID=$3
fi
zmRecord $MONITOR_ID

BTW I'ts not totally understood by how the third argument works:

Syntax: ./zmrecord.sh <Monitor Id> <0|1>(record audio or not) <Linked Monitor Id>
Script will search Linked Monitor for alarm frames, and add count to this monitor.

Seems not working correctly.

Actually what I've understood is:
- the script records chunks in minutes and register it into ZM: works
- should read something from a linked monitor to add what to the recorded high-res stream: not working, even I have a low-res stream in modect
- should maintain the counters of events in the main console page updated: not working.

Thanks for any hint!

Simon

[bbarnett]

Rather than copy+paste your code insert, just download the latest gzip. You can diff against what you have, or just use it, but that latest upload is all my patches + original in one. It's what I use.

Third argument is the zoneminder monitor ID/mid/camera number.

What the linked monitor ID does, is in my first post.

- 'linked monitor' id
- update Events row with 1 alarm frame, if an alarm happens on 'linked monitor'

So if an alarm event happens on the monitor ID you put in the 3rd argument, you'll see alarm frames entered into argument 1's monitor ID.

Counter events should update, but that's not my code.


What I'd suggest here is this ... download the github version, before I did anything. I suspect you'll have the same issues. By the way, you say it doesn't update on the main page, but do you see events when clicking on the monitor events directly?

[xefil]

Goood morning bbarnett,

You mean download latest gzip from github (I've used this https://gist.github.com/mwvent/f4d30599 ... readme-txt) or yours? Yours are only single patches, right? Or do you have a single download file as well?

Ok for the linked monitor function, but seems not working properly in my case. I don't know if due my version (possible) or patches applied wrong.
WAIT: I've found an Alarm Frame Rate set on value "1" on a recorded stream for an event occured on the low-res MoDect, so is that the expected behaviour?

I'll try the original ones as well asap. For now, This is my console:

console.png (19.49 KiB) Viewed 24192 times

You can notice there are no clickable records on the high-res monitor recorded by the script. BTW if I search for them into filters, they are there, clickable and can be shown without issues.

Thanks, Simon

EDIT01:
Found alarm frame, mentioned above

EDIT02:
I've noticed the Monitor_Status was with all event counts set to `(NULL)`. I've tried to stop zm and the script, rewritten all `(NULL)` to `0` and restarted both zm and your script. At the first look, the counters started correctly, setting a first event to `1` with correct size and bandwidth, but some minutes later, back to `(NULL)`. Seems that zm is overwriting those values? But it happens only to the record with monitorid used by your script.

[bbarnett]

Try the script without any of my patches.

I suspect it is the newer version of ZM, but do not know. Further, I likely cannot easily help if so.

By trying the original github latest, you'll eliminate any of my changes as the cause, and instead, zero in on ZM version as the issue.

[xefil]

Thank you anyway bbarnett for your patches which I appreciate very. Due the issue is "only" che console not correctly updated, I'll enable the SQL logging into MySQL to check what ZM is doing. I would like to maintain your patches which i like
If I'll hve news, I'll post them here.

Simon

[xefil]

Hello bbarnett again!
Found the issue, maybe.
I've replaced:

Code: Select all

 SQL="   REPLACE INTO Monitor_Status (MonitorId, Status, CaptureFPS, CaptureBandwidth) 
                VALUES 
                ('$MONITOR_ID','$STATUS', '$FPS', ${LAST_BW:-0})"

with

Code: Select all

SQL="UPDATE Monitor_Status SET Status='$STATUS', CaptureFPS='$FPS', CaptureBandwidth=${LAST_BW:-0} WHERE MonitorId='$MONITOR_ID'";

and it works correctly. AFAIK the `REPLACE` makes a delete and insert.

ZM does following updates on an monitor, i.e. having id 13:

Code: Select all

UPDATE Monitor_Status SET CaptureFPS = 25.17, CaptureBandwidth=77448 WHERE MonitorId=13
INSERT INTO Monitor_Status (MonitorId,AnalysisFPS) VALUES (13, 24.96) ON DUPLICATE KEY UPDATE AnalysisFPS = 24.96

I'll have to reset the whole history maybe, but not so bad.

Hope that helps!

Simon