When you call gst_bin_remove(), the element will be removed from the bin. Since the pipeline took ownership of the element (as it was a newly-created element, see “floating references”) when you added it, and the pipeline drops its reference when you remove it, that means the element will get freed, unless someone else still holds a ref to it. So you can prevent this by adding a gst_object_ref() before removing it.
You can track down these kind of issues (invalid memory access, use-after-free) with valgrind by the way. It will tell you where it got freed.
Having said that, I don’t understand this whole replace-filesink-with-fakesink thing while being paused. I don’t think it makes much sense.
What is the effect you want to have when pausing? If you playback the Matroska file later and you recorded for 10 seconds, paused for 10 seconds, and then recorded again for 10 seconds, how long should the playback be in your opinion? 30 secs or 20 secs? Should it show the paused part from 10-20 secs as ‘freeze frame’ or should it skip it entirely as if it never happened?
Perhaps you want a valve element in front of the encoder/muxer instead to drop data when you’re not recording? But if you want to skip the gaps you would also have to tweak the timestamps with gst_pad_set_offset() on the recording branch tee pad perhaps (and set an offset of -$time_paused whenever you resume recording) or somesuch.
@tpm Even though your solution worked now only the last part of the recording is saved. For example,
Start recording.
Press “Pause” after 10 seconds. (Part A of the recording)
Press “Unpause” after 5 seconds.
Press “Stop” after 40 seconds. (Part B of the recording)
The produced MKV file will contains only part B (40s) of the recording, instead of A + B (50s) as it should.
I tried setting append property of filesink to TRUE before setting pipeline to GST_STATE_PLAYING when user clicks “Unpause”, but in that case only part A (10s) is recorded.
That’s probably because you set the state of the pipeline or branch or elements to NULL and then back to PLAYING. Which doesn’t seem right to me.
Don’t you want to keep seeing the camera feed even when you’re not recording?
Once you send EOS to the recording branch and/or stop writing the file, that file is finished. You can’t start again and add data to it. That’s just not how the file format works. You can’t use append mode here either.
You need to keep the pipeline running all the time, and just not feed data to the recording branch when you don’t want to save it.
So I guess using valve is the best way. I already tried to get it to work with valve, but it didn’t work so I didn’t bother because I thought what I already have will work fine.
Thanks again.
Pausing is now done by setting valve property drop to TRUE and unpausing by setting it to FALSE. If I click “Stop” while the recording is paused (i.e. drop = TRUE) the entire application freezes, but if I continue the recording and then stop the application works normally. So should I always set drop to TRUE when a user clicks “Stop” or is there a better way?
Also, pausing and unpausing produce a corrupted file. MPV gives me these errors.
Playing: Untitled.mkv
(+) Video --vid=1 (*) 'Video' (rawvideo 640x480 30.000fps)
[autoconvert] Converting yuyv422 -> yuv422p
VO: [gpu] 640x480 yuv422p
V: 00:00:00 / 00:00:11 (4%)
[mkv] Corrupt file detected. Trying to resync starting from position 7373405...
V: 00:00:00 / 00:00:11 (8%)
[mkv] Cluster found at 14746363.
V: 00:00:06 / 00:00:11 (58%)
[mkv] Corrupt file detected. Trying to resync starting from position 31335548...
V: 00:00:06 / 00:00:11 (59%)
[mkv] Cluster found at 43009425.
V: 00:00:00 / 00:00:12 (4%)
[mkv] Corrupt file detected. Trying to resync starting from position 7373405...
[mkv] Cluster found at 14746363.
V: 00:00:01 / 00:00:12 (15%)
Exiting... (Quit)
When the file is played it freezes there where I paused the recording.
Again, if I just stop the application the produced file is fine. I guess I should just use gst_pad_set_offset(), as you have pointed out, but how can I get the duration that I need to supply? And can I apply it to the source pad of queue that forwards data to the file?
As for the freezing on stop, try g_object_set(filesink, "async", FALSE, NULL) on the filesink. Without this, it might wait for another preroll buffer when going from PLAYING to PAUSED state when shutting down, and that buffer never comes. Just a guess though.
I don’t know what the cause of the data corruption is, I don’t think it is related to not setting the pad offset. The reason for setting the pad offset is to shift the timestamps forward in front of the muxer after resuming the stream, so that it looks to the muxer like the pausing never happened. Without it, you should just get a long timestamp gap in the file where it keeps showing the last frame, but no data corruption.
Ahh not that one, sorry. But all other (freeze when you click “Stop” when the recording is paused and static frames where the recording was paused) are present.
Application freezes when you click stop while paused: my guess is that the valve drops the EOS event, so it never makes it through to the filesink, so you’re stuck waiting in gst_bus_timed_pop_filtered(bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_EOS);. To fix this, you could set valve drop=false before sending the EOS event, but then you risk a few rogue frames making it through again. It might be easier to also send an EOS event manually to an element or pad after the valve, e.g. the matroskamux sink pad(s). That way all sinks get an EOS event, and then you should also get an EOS message. You can check the GST_DEBUG log to see what’s happening with the EOS events and if they make it to the sinks or not.
That’s what I did. It works. Now I only have to figure out how to use gst_pad_set_offset(). Can you explain to me what to do to get the correct duration to supply as the second argument?
The easiest way might be to have a GTimer which you start whenever you pause the recording and which you pause whenever you unpause the recording. That way the timer will measure the time you spent being paused.
The value you have to pass to gst_pad_set_offset() is in nanoseconds, so you can try passing g_timer_elapsed() * GST_SECOND * -1. For bonus points also handle the microseconds once it works.
You need to find the right pad to set this offset. probably the valve src pad or such. Setting the offset sends SEGMENT events, which should not be dropped by the valve.
This will only work if matroskamux picks up the new segment events properly. I would assume it does but I haven’t checked it.
Thanks again, I’ll try that later because unfortunately I have encountered some more problems.
The actual (complete i.e. with all the capabilities turned on) pipeline doesn’t look like as I have shown until now, nor as I have shown in the example program that I wrote. It actually looks like this:
So it also records audio. I have modified the example program accordingly, click here.
The pipeline is paused by setting drop property of both valve elements to TRUE (and unpausing to FALSE, of course). When a user clicks “Stop” EOS is sent to the entire pipeline and to the matroskamux element.
This new example program has the following problems (none of these are present if you don’t include audio elements):
If you just start the program and click “Stop” the produced file is corrupted.
If you try to stop the program when the recording is paused the program will freeze, hanging on the line that sends EOS to the entire pipeline.
If you click “Pause” and “Unpause” any number of times, as long as the last one was “Unpause” so the recording is not paused, and then click “Stop”, the produced file will contain errors wherever the program was paused.
In all cases the file is playable, though.
Sorry, I know this is not part of the original question but I didn’t expect this, nothing that I tried managed to fix these problem.
For pausing you probably don’t want a valve but the togglerecord element. That ensures that you don’t accidentally drop keyframes or anything else important and basically behaves like a “pause” button on a recorder. It will also not fall apart if you send EOS to the pipeline in paused state, but correctly finish everything at that point then.