Is there a way to split_recording?

[mxmaxime]

Hi,
For an application, I need to split the records in multiple parts (use case: give the user some feedback by sending a video every x seconds).
The important thing: I need to keep the recording during the split, I don't want to loose any frame.

Using the Picamera library, I was using the method split_recording and it was working like a charm.

I didn't find anything that looks similar in picamera2, do you have an idea on how can I do this?
Thank you!

[davidplowman]

Hi, I don't think we have anything quite like that at the moment, so maybe that's one for us to think about. In the meantime there are certainly ways you could do something similar.

I think I might try creating a custom output class and redefining the outputframe method, so something like:

from picamera2.outputs import Output

class MyOutput(Output):

    def outputframe(self, frame, keyframe=True, timestamp=None):
        # Output frame to whatever the current file is. Open a new file every x seconds.

frame is just a buffer of bytes, so you can dump those straight to a file, and timestamp is the timestamp of the frame since the system started, I think it's in nanoseconds. Obviously you can also wait for a keyframe before switching to a new file, and by swapping files within the outputframe method you can be sure not to drop any.

[UltimateCodeWarrior]

So, I was going down a path like this, then I saw in Picamera2 there was an example to just write to a .MP4 file and write audio along with it:

output = FfmpegOutput(mp4_file, audio=True)
picam2.start_recording(encoder, output)

So what I ended up doing was just recording for 5 seconds, and if I detected motion, I would keep pushing back that 5 second limit and in that way I would always have 5 seconds before the motion, and 5 seconds after the motion.

    future = time.time()+record_time

    while time.time() < future :
        lores_frame = picam2.capture_array("lores")
        curr_frame = picam2.capture_array()
        if compare_frames(lores_frame,threshold,current_folder):

When it was time to stop recording this chunk
picam2.stop_recording()
picam.stop_recording()
picam.stop()

Then start the loop all over again with a new mp4 filename to record to. You could always use ffmpeg to stitch them all together.

Now, what is really sad is I haven't yet found a way to split the audio out into frames. In addition to doing motion detection, it would be nice to do audio detection. Maybe you would want to know when the garbage truck was coming down your street so you could get the trash out in time. Or maybe you want to be doing some sort of speech to text transcript. Is there a way whilst recording to do something like curr_audio_chunk = picam2.audio_array() I mean why not? Otherwise, I will have to add a secondary usb microphone and record that to file to do audio monitoring and near real time detection on it.

[davidplowman]

Hi, and thanks for raising this. It's all an interesting question.

As things stand, Picamera2 does no audio processing at all. Audio is simply delegated to the FFmpeg process which inserts it as it writes the mp4 file. So on the one hand, this is a really quick and easy way to add some basic audio functionality (which the original PiCamera didn't have), but it clearly has limitations.

I've often thought about adding "real" audio processing. We'd have to add a thread to read audio packets, and then use some kind of audio encoder. PyAV might be good for this. I'd vaguely imagined one might add all that to the Encoder base class, so that it's largely transparent to applications. Though if you wanted access to the raw audio then one would need something slightly different.

Then the output might be handled again by (another instance of) PyAV. We feed encoded video and audio to the Output class, which would have to be a "PyavOutput" when audio is present, as this would understand audio and video packets (all the other output classes don't). All this would let you do things like store audio into the circular buffers, and write mp4 files from it.

So there's lots to think about, and quite a lot I'd be keen to do. On the downside, this is all running in Python now, not known either for speed or multi-threading efficiency, but I expect it would work OK. More significantly, there's a bit of experimenting and work to do (rpicam-vid does something like this using libav), none of which has ever got high enough on the to-do list. So I'm certainly interested, and open to suggestions, but can't really offer a way forward at this time.

[UltimateCodeWarrior]

The Only thing I saw that might be a Potential lead on doing this was that in the Picamera 2 documentation, on section 9.3. Multiple outputs, maybe there would be a way to output a secondary anorexic video MP4 stream, small pixels, but great audio and then figure out a way how to read the audio out of that near real time.

[jwnxt]

@davidplowman Are there any plans to include the split_recording function in picamera2? It would be very helpful for me as well

[davidplowman]

Hi, I'm afraid I haven't made any progress in that direction. In fact, the next release of Picamera2 will have some more sophisticated output classes like the PyavOutput (with audio support) that I talked about earlier. But in some ways, that actually makes a simple "split_recording" harder!

Assuming you just want to split simple h.264 files, then my first answer is probably still your best bet. Define your own output class, and in the outputframe method, when there's a key frame, switch from one output file to a new one. So it would look a bit like:

from picamera2.outputs import FileOutput

class SplittableOutput(FileOutput):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.split_recording = None

    def outputframe(self, frame, keyframe=True, timestamp=None):
        if keyframe and self.split_recording:
            self.close()
            self.fileoutput = self.split_recording
            sef.split_recording = None
        super().outputframe(frame, keyframe, timestamp)

I haven't tested any of that, but you probably get the idea. At the moment you want to split the recording, do

    output.split_recording = "new_filename.h264"

and then it will happen on the next I frame.

[UltimateCodeWarrior]

Do you have a code example showing how to use the SplittableOutput in conjunction with FfmpegOutput?

I couldn't get this to work, it was throwing an exception:
t_str = str(round(tn,3))
mp4_file = st_utils.get_current_folder()+t_str+".mp4"
fo_MP4 = FileOutput(mp4_file)
so_MP4 = SplittableOutput(fo_MP4)
ffmpeg_MP4 = FfmpegOutput(so_MP4, audio=True)
picam2.start_recording(H264Encoder(), ffmpeg_MP4, quality=Quality.VERY_HIGH)