Video target turns transparent

[Ramblin]

I am not sure how to describe this so please look at the quick video at
https://filedirectory.ayuda.ca/1529-video.mp4

(This is NOT some kind of scam/phishing, ... but if you are unsure, and do not want to click on the link, hopefully the description can suffice)

My recording looks like I am trying out for a sequel to the invisible man.

It starts just fine but quickly degrades to where you can see through me - as if I was turning transparent.

Any idea what is going on?

I have Zoneminder running on a VM under Proxmox (as a check, I Shutdown all other VMs but it makes no difference)
The VM is a fresh Debian 12 with fresh Zoneminder 1.36.33 installed
I added a second 10TB WD Purple drive to hold the videos and the videos are being recorded which I can check by forcing an alarm.
(The transprency effect occurs even when I walk through the camera view after forcing an alarm)

At this point I only have 3 cameras connected and my setup for each is is:
- Monitor A with a low-res (640x360) stream of the camera to do motion detection (modect) with no Frames or Video captured
- Monitor B (same camera) with a hi-res (1920x1080) stream, linked to the low-res stream for triggering the recording at HiRes

I have this setup for all cameras and up until now it has worked fine.

But suddenly I am getting ths transparency effect. Maybe good for an audition in Hollywood but not so good as a security camera.

I have deleted and re-installed both the LoRes and HiRes monitors and it does not fix anything

The actual feed from the camera is NOT the issue; it is being captured to the Mobile app from the manufacturer just fine.
The load on the computer does not seem to be an issue since the other 2 cameras both do fine and the CPU load never goes above 50%.

I have been plaing with the Zone setup for the Modect camera and tried:
- All the Alarm Check Methods: AlarmedPixels FilteredPixels, Blobs
- Both Percentage and Pixels
but it does not make a difference

Any ideas?

[dougmccrary]

I'm afraid the camera is sick, if it was working before, and nothing else changed. Have you restarted it? Have you verified power to it? Were the camera's zm settings loaded from backup of the previous install?
Manufacturer apps can hide flaky frames, so may look good but actually be sub-optimal.

[Ramblin]

@dougmccrary (and others interested)

The camera is working fine. The camera vendor's app on my mobile phone shows perfect recordings but the Zoneminder recording has the transparency effect.

I started over and restored the system to a previous version (it runs on a VM so not a big deal)

I have deleted and re-installed the Camera (both streams) but it did not make a difference

I think I found out what is causing the transparency but I am not sure what to do about it.

When defining a zone using Pixels instead of Percentage, you can specifiy a small portion of the screen. Using percentages, the smallest you can go is 1%

In my situation, my camera is 12 feet above the location being recorded so, with a 640x360 LoRes image (230400 pixels), the size I need to be able to capture is an 800 pixel "blob", which is 0.35 of a percentage.

If I define a larger "blob", the monitor misses some events I want to capture.

But when I get down to really small blobs, the transparency effect kicks in.

So there must be some limit to what the system can handle.

Unfortunately, because of the camera mount location, I need the smaller blob - and the camera vendor app handles the motion detection well.

If anyone can see something I am doing wrong, I'd appreciate suggestions.

[mikb]

Having looked at the video, it does look a bit like video stream corruption -- the way H264/265 and similar files are compressed does a lot of reference to forward/backward-in-time key-frames, and when the data gets messed up, that's the kind of breakup you get.

Are you recording a direct camera pass-through of the video stream, or is Zoneminder processing into JPEGs to store it?

If pass-through: What happens if you find the raw file in your events directory (you will need to search for it) and play that in VLC or similar player?

[Ramblin]

Are you recording a direct camera pass-through of the video stream, or is Zoneminder processing into JPEGs to store it?

If pass-through: What happens if you find the raw file in your events directory (you will need to search for it) and play that in VLC or similar player?

I am using Direct Pass-Through for the HiRes Camera stream (No Encoding) and have tried various settings on the LoRes camera stream, including Disabled for both JPEG and Video but none of that seems to make a difference.

I did download the actual mp4 file from the Zoneminder system onto my PC and play it with VLC. It showed the transparency effect just like it did when played via the Zoneminder GUI. The file you are viewing is a the one I downloaded and stored on a server for you to access. Is the mp4 file from the events directory the "raw" file you were referrring to or is there another?

An unrelated question: Do you know how I get emails when a post I have made has a reply? I am not getting notifications and only see Replies when I log back in and check. I am "subscribed" to this post (be default, when I create it" and my User Configurations says I should be getting notifications, but it is not happening.

[dougmccrary]

Do you know how I get emails when a post I have made has a reply?

They're supposed to go to your registered email address.
I just sent one - if you don't get it, maybe re-register?

[Ramblin]

Do you know how I get emails when a post I have made has a reply?

They're supposed to go to your registered email address.
I just sent one - if you don't get it, maybe re-register?

I received your email and replied to it

Thank you for following up

I am not sure if what I have discovered is a bug or if the system was never intended to get to the low level of detection I need.
If it is a bug, how do I know and if it is, how do I submit a bug report?

[mikb]

Are you recording a direct camera pass-through of the video stream, or is Zoneminder processing into JPEGs to store it?

If pass-through: What happens if you find the raw file in your events directory (you will need to search for it) and play that in VLC or similar player?

I am using Direct Pass-Through for the HiRes Camera stream (No Encoding)

Is the mp4 file from the events directory the "raw" file you were referrring to or is there another?

I was trying to work out whether the camera was providing bad data to ZM, or whether ZM was getting good data but mangling it on playback. By the "raw event" I mean going around ZM and finding the file in the filesystem under your storage location, under the folder for that camera ID at that date and time. The direct video from the camera will be in an mp4 file there (hence pass through).

If *that* shows the corruption, it seems like the camera is returning corrupted data. If that file is good, but when you view it/download it *through* ZM it's corrupted, then ZM is corrupting it.

By downloading it through ZM, exporting it from ZM, viewing it in ZM, you can't be sure whether the fault is one side or the other.

[iconnor]

If your system can't keep up, the queues will fill and then ZM will start dropping frames. Since we always keep a keyframe in the queue, you can get a situation where the video starts with a full image and then you get crap for the rest.

Are there warnings in your logs about this?

[Ramblin]

By downloading it through ZM, exporting it from ZM, viewing it in ZM, you can't be sure whether the fault is one side or the other.

Interesting

I diod not realize there would be a difference between videos downloaded with the Zoneminder GUI anda direct download from the direcotry

But there was

Both show the transparency efefct, but they are not identical; one shows at a slightly different sequence/time of the transparency than the other.

[Ramblin]

If your system can't keep up, the queues will fill and then ZM will start dropping frames. Since we always keep a keyframe in the queue, you can get a situation where the video starts with a full image and then you get crap for the rest.

Are there warnings in your logs about this?

Thanks for replying Isaac

And please accept my thanks for sharing such a great system.

How do I tell if it is a case of the computer being overloaded?

I do not think that is the case since I monitor the CPU load and it never goes over 50%. I do not have a graphic card so it is all CPU loaded video.

I tried it again tonight (past dark so just B&W) and ?luckily enough? it did the same thing so I checked the logs

I looked in the log at /var/log/zm/zmc_m6.log (Monitor 6). Is this the log you were thinking about?

There are errors

Code: Select all

11/20/23 17:53:04.397642 zmc_m6[708].INF-zm_monitor.cpp/1680 [Back Yard HiRes: 5300 - Capturing at 14.97 fps, capturing bandwidth 32425bytes/sec Analysing at 0.00 fps]
11/20/23 17:53:11.078846 zmc_m6[708].INF-zm_monitor.cpp/1680 [Back Yard HiRes: 5400 - Capturing at 14.97 fps, capturing bandwidth 33245bytes/sec Analysing at 0.00 fps]
11/20/23 17:53:16.635548 zmc_m6[722].INF-zm_monitor.cpp/1976 [Back Yard HiRes: 5484 - Gone into alarm state PreAlarmCount: 0 > AlarmFrameCount:1 Cause:Linked]
11/20/23 17:53:16.740928 zmc_m6[722].INF-zm_monitor.cpp/1983 [Back Yard HiRes: 5483 - Opening new event 130, alarm start]
11/20/23 17:53:16.836531 zmc_m6[780].INF-zm_videostore.cpp/532 [some options not used, turn on debugging for a list.]
11/20/23 17:53:17.197899 zmc_m6[722].INF-zm_monitor.cpp/2016 [Back Yard HiRes: 5491 - Gone into alert state]
11/20/23 17:53:17.198296 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:17.718735 zmc_m6[708].INF-zm_monitor.cpp/1680 [Back Yard HiRes: 5500 - Capturing at 15.06 fps, capturing bandwidth 41123bytes/sec Analysing at 0.00 fps]
11/20/23 17:53:17.916481 zmc_m6[722].INF-zm_monitor.cpp/2000 [Back Yard HiRes: 5502 - Gone back into alarm state]
11/20/23 17:53:18.635545 zmc_m6[722].INF-zm_monitor.cpp/2016 [Back Yard HiRes: 5513 - Gone into alert state]
11/20/23 17:53:19.197978 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:19.517192 zmc_m6[722].INF-zm_monitor.cpp/2000 [Back Yard HiRes: 5526 - Gone back into alarm state]
11/20/23 17:53:20.997884 zmc_m6[722].INF-zm_monitor.cpp/2016 [Back Yard HiRes: 5548 - Gone into alert state]
11/20/23 17:53:21.197700 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:23.197870 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:23.917694 zmc_m6[722].INF-zm_monitor.cpp/2000 [Back Yard HiRes: 5592 - Gone back into alarm state]
11/20/23 17:53:24.397959 zmc_m6[708].INF-zm_monitor.cpp/1680 [Back Yard HiRes: 5600 - Capturing at 14.97 fps, capturing bandwidth 35675bytes/sec Analysing at 0.00 fps]
11/20/23 17:53:25.199298 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:27.198007 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:28.398964 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:28.598472 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:28.797711 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:28.998428 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:29.197884 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:29.398718 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:31.084496 zmc_m6[708].INF-zm_monitor.cpp/1680 [Back Yard HiRes: 5700 - Capturing at 14.96 fps, capturing bandwidth 40439bytes/sec Analysing at 0.00 fps]
11/20/23 17:53:31.198152 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:32.198015 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:32.397803 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:32.599240 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:32.797795 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:32.998510 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:33.197862 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:33.398849 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:34.398484 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:35.197840 zmc_m6[780].ERR-zm_videostore.cpp/1425 [Error writing packet: Invalid argument]
11/20/23 17:53:35.998538 zmc_m6[722].INF-zm_monitor.cpp/2016 [Back Yard HiRes: 5773 - Gone into alert state]
11/20/23 17:53:36.317374 zmc_m6[722].INF-zm_monitor.cpp/2024 [Back Yard HiRes: 5778 - Left alarm state (130) - 318(222) images]
11/20/23 17:53:36.317448 zmc_m6[722].INF-zm_monitor.cpp/2029 [Back Yard HiRes: 5778 - Closing event 130, alarm end]
11/20/23 17:53:37.477975 zmc_m6[722].INF-zm_monitor.cpp/1976 [Back Yard HiRes: 5796 - Gone into alarm state PreAlarmCount: 0 > AlarmFrameCount:1 Cause:Linked]
11/20/23 17:53:37.636776 zmc_m6[708].INF-zm_monitor.cpp/1680 [Back Yard HiRes: 5800 - Capturing at 15.26 fps, capturing bandwidth 48090bytes/sec Analysing at 0.00 fps]

I do not know what these errors mean nor how to fix them.

I am wondering if the "invalid packet" is because I have the number of Pixels in the Blob/Filtered Area fewer than it would take to be 1% of the Zone area. If I use Percentages, it will not let me put less than 1% (and even at 1%, it starts acting up). The issue starts occuring when I set that number low. The issue goes away when I make that number larger, but then I don't detect the things I want to detect. I expect this is because I have the camera mounted 12 feet (actually more like 15 feet) above ground level so the size of the area I need to use to capture alarms is small.

Do they answer any questions for you and if so, can you suggest something I do ?

[Ramblin]

If your system can't keep up, the queues will fill and then ZM will start dropping frames.

I wanted to wait until I had time to experiment and have now tried a few things to troubleshoot.

I tried this during the day in case there was an issue with InfraRed imaging.
I disabled all other cameras (there are only 4 but to be sure)

I ran the test with the Alarm Check Method set to Blobs
I monitored the CPU usage throughout and it never went above 25%
The "transparency" effect (I do not know what else to call it) was still present in Zoneminder's video
The vendor's app on my phone recorded clean video
So it is not a camera issue

I simplified the CPU usage and set the Alarm Check Method to Filtered Pixels
I monitored the CPU usage throughout another event and it never went above 15%
Same as above: vendor's video was clean (so camera is working); Zoneminders video was as before

The only common element is that the Alarm/Filter area is less than 1% of the Zone area (about .4%)

Is there any chance that there is an issue in software when the Alarm/Filter area gets really low - other than CPU load?
Is there another "load" metric I should be using to see if it is a load issue?
I have monitored CPU, DIsk I/O and Network traffic. So far no issue with any.

[iconnor]

In passthrough mode, all video is effectively read-only. We take it in, we write it to disk.

Zones should have no effect on anything. However you say this only starts after getting down to a really small zone... I'm thinking red herring but...
What about if you turn off motion detection and just record?

I have seen lots of cameras (reolink mainly) that do this because they interleave the packets too much and I think ffmpeg gives up on them after a while. You can improve it by using udp with a large reorder_queue_size. Or you can use an rtmp stream instead.

But you say it only happens with a small zone. Doesn't make sense.

So you need to turn on debug. And we need to double down on verifying that zones config causes it. Cuz it shouldn't.

[Ramblin]

you say this only starts after getting down to a really small zone... I'm thinking red herring but...
What about if you turn off motion detection and just record?

First off, apologies for not replying earlier; I am not getting notifications when a reply is made to my post and I came back in today to see if there had been any and here you are.

I really appreciate you helping out here. If I can get this resolved, I do intend to set this up as my default security system and I will make a donation.

I will try turning off Motion detection and see what that does.

Just to clarify, and it may be that we are saying the same thing, but it is not making the Zone smaller that causes the issue but is instead making the settings in the zone:
- Alarm Area
- Filtered Area
- Blob Area
When these become such a small % of the Zone, that is when the "transparency effect" kicks in.

I see you are in Toronto, so you are about to get the same weather as me here in Kitchener. So today is not going to be a good day for testing outdoor cameras, but I will reply soon.

Thanks again.

[iconnor]

The only thing I can think of is that the really small zone settings cause it to detect too many blobs, and just spend too much time doing motion detection, sucking up cpu.

If that is the case, then your cpu should be maxed out, there should be lots of errors in the logs, etc..

Looks like the snow is here for at least a week....