Broken Database

[watcherwatcher]

I've gotten myself in quite the predicament here. I moved the storage from the default location of /var/cache/zoneminder/events into a /cctv directory which is mounted to a SAN. After I did that, I got rid of all of the archived footage that I didn't need. I don't remember exactly how I did this, I might have just "rm -fR"ed the directory to free up space. Whatever I did, it was obviously wrong because now I'm getting many errors like this:

Can't open /var/cache/zoneminder/events/3/2019-08-18/17936/00005-capture.jpg: No such file or directory

These are coming from zm_eventstream.cpp:701 so I looked at the source code to see where it's getting this path, and why on Earth it is looking at an event from last August! I found filepath is set on line 654 using an snprintf. This code is in the sendFrame method. Why is it sending a frame from this old video when nobody is watching any old files? Great question. I don't know.

The "event_data->path" seems to be set on line 209 in loadEventData(). Again, I don't know what we're loading event data, but I see storage_path comes from storage->Path(). When I've seen this convention before, it's mapped directly to database tables, so I decided to look there:

Code: Select all

mysql> SELECT Id, Path, Type, Scheme FROM Storage;
+----+-------+-------+--------+
| Id | Path  | Type  | Scheme |
+----+-------+-------+--------+
|  2 | /cctv | local | Medium |
+----+-------+-------+--------+
1 row in set (0.00 sec)

mysql> SELECT DISTINCT StorageId FROM Events;
+-----------+
| StorageId |
+-----------+
|         2 |
+-----------+
1 row in set (0.00 sec)

mysql> SELECT * FROM Events WHERE Id = 17936;
Empty set (0.00 sec)

The event doesn't exist. So my system is attempting to process an event that isn't in the database, using a path that isn't in the Storage table in the database, the source code, nor the config file (/etc/zm/zm.conf). The only clue I have to go on is what the web Console tells me. In the Archived column, it has zeros for all of the cameras, however it only shows 0B for two of the cameras. Two of the cameras are showing 0 entries at 92.75MB and 87.49MB. This appears to just be coming from the Monitors table:

Code: Select all

mysql> SELECT ArchivedEventDiskSpace FROM Monitors;
+------------------------+
| ArchivedEventDiskSpace |
+------------------------+
|                      0 |
|                      0 |
|               97254167 |
|               91738712 |
+------------------------+
4 rows in set (0.00 sec)

I could just zero that out with an UPDATE statement, but I figured since I'm posting here asking for help, maybe I should wait on that before I end up causing more problems.

This is where my trail ends. I'm really not sure where to go from here. If anyone can tell me what I can troubleshoot next in order to track this down, I'd be very appreciative. I'm running v1.34.15.

[watcherwatcher]

Apparently rebooting was the answer. If this happens to you, now you know what to do.