Problem with stored events when disconnecting and reconnecting storage

[IgorA100]

There is a separate storage that is mounted on the Ubuntu system and is used in ZM.
At some point, I simply disconnected this storage without unmounting it and without stopping ZM (we will consider it an emergency).
After turning on the storage, problems appeared:
1. The monitors still have the same storage indicated, but they don’t write to it! If you go into the monitor settings and resave, recording begins. Rebooting ZM does not help, only resaving the monitor settings.
2. Old records of events become a ghost, because they are not visible in the event list, but take up disk space. This is not good, because... On the one hand, there are events, but on the other hand, I can’t see them and they are not cleared by the filter!
I might have missed some of the nuances, but the essence of the problem is there.

[IgorA100]

My guess: When cleaning, the filter does not check the availability of the file being deleted and/or the fact that the file was deleted. Those. The database is being cleaned, but in reality the file is not deleted, because he was not available.
But I could be wrong, because... I didn't look at the code.

[IgorA100]

After looking a little at the code in “sub delete { -> \ZoneMinder\Event.pm” my thought was confirmed. First, the record is deleted from the database, and then the file is deleted. It's probably worth deleting the file first and only if the event file is successfully deleted, delete the entry from the database. It is also worthwhile to provide for “database repair”, when, for example, the event file was actually deliberately deleted and so that the entry in the database does not hang forever.

[iconnor]

The logic goes both ways. Some people just want the event to disappear and get deleted later in the background. When you get into s3fs filesystems, etc the performance of deleting is so bad, that you just want to delete from the db anyways.

We take the view that the video has the greatest value, and so should be deleted last.

In the event of this kind of situation, we have zmaudit.pl which goes through and cleans things up. It used to run all the time, but it is too much of a resource hog, so now we only recommend running it manually or from cron.

[IgorA100]

We take the view that the video has the greatest value, and so should be deleted last.

I completely agree with this!

performance of deleting is so bad, that you just want to delete from the db anyways.

That's right. But the cleaning itself when using the filter is already performed in the background, so additional analytics would not greatly affect the overall speed of ZM. Let it run quietly in the background.
But for this implementation you will have to rewrite a lot of things and there may be nuances during the debugging process. I would like to first sketch out an algorithm for such work, but I still have problems with time

we have zmaudit.pl which goes through and cleans things up

Great! Somehow I didn't even know about him
Launched, deletes something

[IgorA100]

The main disadvantage of the current cleaning algorithm using a filter is that it is temporary!!! storage unavailability:
- The files remain, but they cannot be viewed in ZM even after connecting the storage.
- Data is deleted from the database, but in reality there is no more space.

Those. you need to backup all the data, and then deal with it. What if there are thousands of events???
In the case of a permanent storage shutdown, of course, everything is logical.

[iconnor]

You can add an exists in filesystem rule to only delete accessible events.

[IgorA100]

Oh yes, there really is such an option.
That's great, I'll test it.

[IgorA100]

After running zmaudit.pl, storage space was cleared.
ZM shows 8-9% of used space, this seems to be true, but the filter now does not work correctly. Those. more than 9% do not want to use it. The filter continues to delete recent events even though there is enough storage space.
Strange behavior. Before reconnecting the storage everything worked fine.

[IgorA100]

I'm confused...
It turns out that events from a storage that was reconnected are deleted by a completely different filter, which is configured for the default storage location.
How can this be?

[IgorA100]

I don't understand anything at all...
I had one monitor whose recording settings indicated: Storage Area = Unspecified, and it recorded to the default storage.
When I changed his settings to Storage Area = Default, then all storage began to be cleared correctly.
There must have been some kind of glitch...

[dougmccrary]

I've seen this.
The fault before you change it is NULL for default. If you set it default, it's 0. Or whatever your default storage ID is.