| Liz: you’ll notice that this post has no pictures or video. That’s because we’d like you to make some for us, using the new camera mode. Take some 90fps video using our camera board and the information below, slow it down to 30fps and send us a link: if yours is particularly splendid, we’ll feature it here and on the front page. Over to JamesH! When the Raspberry Pi camera was released, the eagle-eyed among you noticed that the camera hardware itself can support various high frame rate modes, but that the software could ‘only’ manage 30 frames per second in its high-definition video mode. There’s is no hardware limitation in the Raspberry Pi itself. It’s quite capable of handling these high frame rate modes, but it does require a certain amount of effort to work out these new ‘modes’ inside the camera software. At the original release of the camera, two modes were provided: a stills capture mode, which offers the full resolution of the sensor (2592×1944), and a 1080p video mode (1920x1080p). Those same eagle-eyed people will see that these modes have different aspect ratios – the ratio of width to height. Stills outputs 4:3 (like 35mm film), video 16:9 (wide screen). This creates a problem when previewing stills captures, since the preview uses the video mode so it can run at 30 frames per second (fps) – not only is the aspect of the preview different, but because the video mode ‘crops’ the sensor (i.e. takes a 1920×1080 windows from the centre), the field of view in preview mode is very different from the actual capture. We had some work to do to develop new modes for high frame rates, and also fix the stills preview mode so that is matches the capture mode. So now, finally, some very helpful chaps at Broadcom, with some help from Omnivision, the sensor manufacturer, have found some spare time to sort out these modes, and not just that but to add some extra goodness while they were at it. (Liz interjects: The Raspberry Pi Foundation is not part of Broadcom – we’re a customer of theirs – but we’ve got a good relationship and the Foundation’s really grateful for the volunteer help that some of the people at Broadcom offer us from time to time. You guys rock: thank you!) The result is that we now have a set of mode as follows :
I’ve introduced a new word in the that list. Binned. This is how we can get high frame rates. Binning means combining pixels from the sensor together in a ‘bin’ in the analogue domain. As well as reducing the amount of data, this can also improve low light performance as it averages out sensor ‘noise’ in the absence of quantisation noise introduced by the analogue to digital converters (ADCs), which are the bits of electronics in the sensor that convert the analogue information created by incoming photons to digital numbers. So if we do a 2×2 ‘bin’ on the sensor, it only sends a quarter (2×2 = 4 = a quarter!) of the amount of data per frame to the Raspberry Pi. This means we can quadruple (approximately – there are some other issues at play) the frame rate for the same amount of data! So a simple 2×2 bin theoretically means quadruple the frame rate, but at half the X and Y resolution. This is how the 1296×972 mode works – it’s exactly a 2×2 binned mode, so it’s still 4:3 ratio, uses the whole sensor field of view, and makes a perfect preview mode for stills capture. We also have a very similar mode, which is 1296×730. This is used for 720p video recording (the sensor image is scaled by the GPU to 1280×720). This is a 2×2 binned mode with an additional crop, which also means a slightly increased frame rate as there is less data to transfer. Now by reducing the resolution output by the sensor even further and by using ‘skipping’ of pixels in combination with binning, we can get even higher frame rates, and this is how the high speed 640×480 VGA modes work. So, the fastest mode is now VGA resolution at 90 frames per second – three times the frame rate of 1080p30. So, how do we use these new modes? The demo applications One thing to note: the system will always try to run at the frame rate specified in preference to resolution. Therefore if you specify a high rate at a resolution it cannot manage, it will use a low resolution mode to achieve the frame rate and upscale to the requested size – upscaling rarely looks good. It may also be too fast for the video encoder, so some of the extra frames may be skipped. So always ensure the resolution you specify can achieve the required frame rate to get the best results. So, a quick example, to record a 10s VGA clip at 90fps There have also been minor changes to the V4l2 driver to support these new modes. These should be included when you do the The V4L2 driver supports the new modes too. Just using the normal requests, you can now ask for up to 90fps. So doing the same streaming of VGA at 90fps to H264 would be the following set of v4l2-ctl commands: There are a few provisos that you will need to consider when using the faster modes, especially with the V4L driver.
That said, most people should find no problem with these new features, so a big thank you must go to Dave Stevenson and Naush Patuck at Broadcom for finding the time to implement them! Also, thanks to Omnivision for their continued support. |
Tuesday, March 18, 2014
New camera mode released
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment