Huge Database

[juanmoura]

I was checking that even using Storage Areas, and scoring for H264 Passthroug the system keeps event frames within the database.
Is there any way to delete these frames since I have the recordings?

I see, in the events directory are the videos only, but inside the bank are the frames in the resolution that I enter in Zoneminder.

Catching on a huge database. I would like to see a way to delete and stop recording those unnecessary images on the bench.

[rockedge]

perhaps using some Filters to do this?

[juanmoura]

no no.. i just check the path /etc/var/cache/zoneminder/events/1/1000/1000-video.mp4
don't have any frame here

But my database have 36gb (just the mysql database)

I make a backup from mysql and put in other machine and i see all frames.

Inside de database have Monitors,Storage......Frames
when the zoneminder create a mp4 (h264 passthroug) it put inside the database all frames..

I try in other VMs and the result is the same

[juanmoura]

Hunting a lot and researching found what is leaving my database so bulky.
As I said earlier, I have seen that it is being stored in the database within the FRAMES table the frames-by-frame images in relation to the second that has movement.
This image is unnecessary since I am saving it to MP4, and consequently if it stops storing the frames with it, so reduce the size of the database and also increase the performance of the database, either on the same machine or in an external database.

Searching I saw that the allocation of the frames in the database is done by the BULK_FRAME_INTERVAL option, so I ask, how do I disable this function? By default it comes configured with 100, I have tested changing it to 0, but it continues to write to the frames database

[Alec]

. . . the allocation of the frames in the database is done by the BULK_FRAME_INTERVAL . . . By default it comes configured with 100, I have tested changing it to 0, but it continues to write to the frames database

The BULK_FRAME_INTERVAL is designed to reduce the size of the database. The number that you set (e.g. the default of 100) determines how many frames are recorded per record in the database. So, if you have 10,000,000 frames with the default setting of 100, you will have 100,000 records (plus some extras at the beginning and end of the event). If you increase the setting to 1000, you will only have 10,000 records (plus that same number of extras at the beginning and end of the event).

If I understand you correctly, I think that increasing the BULK_FRAME_INTERVAL will help address your problem.

[juanmoura]

I will perform this test, but even if I increase the value it will continue to store the images in the database. In fact my intention is not to save smaller amount, is to eliminate this frame recording that has no need once the event has already been recorded on the disc in mp4.

It is a busy space without necessity and a process created without necessity.

The way it is it saves the images in MP4 and saves the frames in jpg in the database.

[Alec]

Enter the monitor configuration, select the "Storage" tab, and change the setting for "Save JPEG".

[juanmoura]

No Alec, understand, I'm already recording to MP4, so I do not want to record in JPG.

On storage tab is selected only for H264 Passthroug.

I'm trying to make it stop recording both in JPG inside FRAMES table and MP4.

Speaking in a more amateurish way, I just want to see to eliminate the FRAMES table because it is unnecessary.

For that I will see the photos at 320x240 (the resolution I put in the zoneminder) if I can see the recordings at 1920x1080 (which is the resolution recorded via H264 Passthroug) ????

Understood? My English is not very good. If you do not understand, let me know.

[Alec]

When you set up the monitor, under the storage tab you can choose what frames are saved as jpegs and also to save the H264 pass through. It sounds like you are set to save both the H264 stream AND JPEG frames.

Also, if you view the frames of a saved event, even if you are not set to save JPEGS, the frames will be extracted from the mpeg file and saved in the event folder.

[juanmoura]

My setting is

SAVE JPEG DISABLE

VIDEO WRITER - H264 Passthroug

and the server save in the /var/cache/zoneminder/events/ saves only .mp4

but inside the database, inside the table FRAMES, are saving JPEGS

Find in your system when you open a video backup the option FRAMES.

In there are many many jpgs.

[juanmoura]

https://imgur.com/a/cnWh5

See the option FRAMES

[Alec]

Code: Select all

+----------+---------+--------+
| Id       | FrameId | Type   |
+----------+---------+--------+
| 22684336 |       1 | Normal |
| 22684337 |       2 | Normal |
| 22684338 |       3 | Normal |
| 22684339 |       4 | Normal |
| 22684345 |       5 | Normal |
| 22684348 |       6 | Normal |
| 22684352 |       7 | Normal |
| 22684355 |       8 | Normal |
| 22684359 |       9 | Normal |
| 22684366 |      10 | Normal |
| 22684369 |      11 | Normal |
| 22684372 |      12 | Normal |
| 22684376 |      13 | Normal |
| 22684380 |      14 | Normal |
| 22684386 |      15 | Normal |
| 22684395 |      16 | Normal |
| 22684396 |      17 | Normal |
| 22684397 |      18 | Normal |
| 22684401 |      19 | Normal |
| 22684405 |      20 | Normal |
| 22684408 |      21 | Normal |
| 22684414 |      22 | Normal |
| 22684417 |      23 | Normal |
| 22684421 |      24 | Normal |
| 22684426 |      25 | Normal |
| 22684432 |     100 | Bulk   |
| 22684436 |     200 | Bulk   |
| 22684440 |     300 | Bulk   |
| 22684444 |     400 | Bulk   |
| 22684448 |     500 | Bulk   |
| 22684452 |     600 | Bulk   |
| 22684457 |     606 | Normal |
+----------+---------+--------+

These are all of the frame records in my database for a random continuous event.

The event is 606 frames long, but there are only 32 records in the Frames table as there are 6 "Bulk" frame entries that are (up to) 100 frames long.

If you select the frames option that you noted above when viewing an event, the frames will be extracted from the MPEG file, but they are not created until you view them.

[knight-of-ni]

The best person to answer this question is iconnor. I have not tested the newer videowriter code extensively myself, but the size of your database does not sound normal. I certainly don't recall any of my tables growing significantly the times I have tested it. Only on very large or very long running systems would I expect the database itself to grow over 1GB.

Try getting his attention in the zoneminder irc channel. He goes by ic0n there.

[juanmoura]

Alec, I'm sorry but the statement you made is wrong. Come on, my setup basically consists of:

1 Camera, set the resolution in Zoneminder as 640x480, however the resolution on the camera is 1920x1080 ...
In the storage tab this is disabled to save JPEG and enabled H264 Passthroug

That way when I click on the monitor I watch in 640x480, but when it recognizes the motion and records and I open the recorded file it is at 1920x1080p

This way I can save ram memory by putting more cameras.

When I click on the FRAMES tab I do not see a extracted MP4 frame, I see a extracted frame with the resolution of 640x480 even without I see.

What I mean is that, my system has 43 cameras with 12 tb, all 2MP, set as MODETEC. The server started operating in mid-November 2017, and now it is at 52% space in disc.

I was doing a backup of the database to separate from the server using multiservers and I came across this situation, 36gb database, I went back and saw that it was the table of frames storing photos that are unnecessary.

When the zoneminder is operating the resource quantity for mysql is large and the more unnecessary processes the less resources we will have for the system.

In the log I see it creating the images, later creating the .MP4 and later it deleting the JPEG images created, but it backs up inside the bank without necessity.

Why save frames if I already own the mp4 file?

Removing this from the system will improve performance.

[juanmoura]

Iconnor, any help?