You might want to have a look at the camerabin element, it can take care of all the low-level details for you.
Some example code here: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/tree/master/tests/examples/camerabin2 (might be a bit bitrotten, but shouldn’t be too hard to fix if so).
I have found a fix for problems 1 and 3. All I had to do is send EOS event and wait for it before I set pipeline to GST_STATE_NULL. Fix can be found here.
Now I only have to figure out how to solve second problem.
Element 0x1043a770 in Matroska is Chapters, which isn’t mandatory. The error message comes from this line in mpv but I still can’t figure out when it happens.
You can try checking the file with mkvalidator, or comparing the good and bad files with mkvtoolnix (the “Info tool” tab on the left).
That seems like an overkill for my purposes. And I have only one problem left, I’m not going to rewrite 1000 LOC now.
Ok, now I have only this problem. Please see related code. You can try setting a brakpoint on line 199.
The problem is that now when I try to unpause pipeline.sink_file fails to be recognized as GObject for some reason.
When you call gst_bin_remove(), the element will be removed from the bin. Since the pipeline took ownership of the element (as it was a newly-created element, see “floating references”) when you added it, and the pipeline drops its reference when you remove it, that means the element will get freed, unless someone else still holds a ref to it. So you can prevent this by adding a gst_object_ref() before removing it.
You can track down these kind of issues (invalid memory access, use-after-free) with valgrind by the way. It will tell you where it got freed.
Having said that, I don’t understand this whole replace-filesink-with-fakesink thing while being paused. I don’t think it makes much sense.
What is the effect you want to have when pausing? If you playback the Matroska file later and you recorded for 10 seconds, paused for 10 seconds, and then recorded again for 10 seconds, how long should the playback be in your opinion? 30 secs or 20 secs? Should it show the paused part from 10-20 secs as ‘freeze frame’ or should it skip it entirely as if it never happened?
Perhaps you want a valve element in front of the encoder/muxer instead to drop data when you’re not recording? But if you want to skip the gaps you would also have to tweak the timestamps with gst_pad_set_offset() on the recording branch tee pad perhaps (and set an offset of -$time_paused whenever you resume recording) or somesuch.
Thanks, it works.
It should be 20 seconds, as if it never happened.
Thanks, I’ll check out valve element.
@tpm Even though your solution worked now only the last part of the recording is saved. For example,
The produced MKV file will contains only part B (40s) of the recording, instead of A + B (50s) as it should.
I tried setting append property of filesink to TRUE before setting pipeline to GST_STATE_PLAYING when user clicks “Unpause”, but in that case only part A (10s) is recorded.
That’s probably because you set the state of the pipeline or branch or elements to NULL and then back to PLAYING. Which doesn’t seem right to me.
Don’t you want to keep seeing the camera feed even when you’re not recording?
Once you send EOS to the recording branch and/or stop writing the file, that file is finished. You can’t start again and add data to it. That’s just not how the file format works. You can’t use append mode here either.
You need to keep the pipeline running all the time, and just not feed data to the recording branch when you don’t want to save it.
So I guess using valve is the best way. I already tried to get it to work with valve, but it didn’t work so I didn’t bother because I thought what I already have will work fine.
Thanks again.
I managed to almost do it with valve.
If I just record and press “Stop” it works fine.
Pausing is now done by setting valve property drop to TRUE and unpausing by setting it to FALSE. If I click “Stop” while the recording is paused (i.e. drop = TRUE) the entire application freezes, but if I continue the recording and then stop the application works normally. So should I always set drop to TRUE when a user clicks “Stop” or is there a better way?
Also, pausing and unpausing produce a corrupted file. MPV gives me these errors.
Playing: Untitled.mkv
(+) Video --vid=1 (*) 'Video' (rawvideo 640x480 30.000fps)
[autoconvert] Converting yuyv422 -> yuv422p
VO: [gpu] 640x480 yuv422p
V: 00:00:00 / 00:00:11 (4%)
[mkv] Corrupt file detected. Trying to resync starting from position 7373405...
V: 00:00:00 / 00:00:11 (8%)
[mkv] Cluster found at 14746363.
V: 00:00:06 / 00:00:11 (58%)
[mkv] Corrupt file detected. Trying to resync starting from position 31335548...
V: 00:00:06 / 00:00:11 (59%)
[mkv] Cluster found at 43009425.
V: 00:00:00 / 00:00:12 (4%)
[mkv] Corrupt file detected. Trying to resync starting from position 7373405...
[mkv] Cluster found at 14746363.
V: 00:00:01 / 00:00:12 (15%)
Exiting... (Quit)
When the file is played it freezes there where I paused the recording.
Again, if I just stop the application the produced file is fine. I guess I should just use gst_pad_set_offset(), as you have pointed out, but how can I get the duration that I need to supply? And can I apply it to the source pad of queue that forwards data to the file?
As for the freezing on stop, try g_object_set(filesink, "async", FALSE, NULL) on the filesink. Without this, it might wait for another preroll buffer when going from PLAYING to PAUSED state when shutting down, and that buffer never comes. Just a guess though.
I don’t know what the cause of the data corruption is, I don’t think it is related to not setting the pad offset. The reason for setting the pad offset is to shift the timestamps forward in front of the muxer after resuming the stream, so that it looks to the muxer like the pausing never happened. Without it, you should just get a long timestamp gap in the file where it keeps showing the last frame, but no data corruption.
I think you need to post your latest code again.
No, that didn’t help.
Ok. Still, how can I get the duration that I need to supply as the second argument to gst_pad_set_offset()?
The valve element should be before the encoder/muxer, not before the file sink.
Perhaps you could be more specific than “it doesn’t work”? What did you change and what was the outcome? Do you get an error, etc?
I changed the pipeline from
/-> queue -> autovidesink
v4l2src -> videoconvert -> queue -> tee
\-> queue -> matroskamux -> queue -> valve -> filesink
to
/-> queue -> autovidesink
v4l2src -> videoconvert -> queue -> tee
\-> queue -> valve -> matroskamux -> queue -> filesink
But the outcome doesn’t change: all problems described here still exist.
All of them? Even the matroska data corruption where it says re-syncing to next cluster?
Ahh not that one, sorry. But all other (freeze when you click “Stop” when the recording is paused and static frames where the recording was paused) are present.
Application freezes when you click stop while paused: my guess is that the valve drops the EOS event, so it never makes it through to the filesink, so you’re stuck waiting in gst_bus_timed_pop_filtered(bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_EOS);. To fix this, you could set valve drop=false before sending the EOS event, but then you risk a few rogue frames making it through again. It might be easier to also send an EOS event manually to an element or pad after the valve, e.g. the matroskamux sink pad(s). That way all sinks get an EOS event, and then you should also get an EOS message. You can check the GST_DEBUG log to see what’s happening with the EOS events and if they make it to the sinks or not.