[RFC] Resolution change support in video codecs in v4l2

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

Yesterday we had a chat about video codecs in V4L2 and how to change the
interface to accommodate the needs of GStreamer (and possibly other media
players and applications using video codecs).

The problem that many hardware codecs need a fixed number of pre-allocated
buffers should be resolved when gstreamer 0.11 will be released.

The main issue that we are facing is the resolution change and how it should be
handled by the driver and the application. The resolution change is
particularly common in digital TV. It occurs when content on a single channel
is changed. For example there is the transition from a TV series to a
commercials block. Other stream parameters may also change. The minimum number
of buffers required for decoding is of particular interest of us. It is
connected with how many old buffers are used for picture prediction.

When this occurs there are two possibilities: resolution can be increased or
decreased. In the first case it is very likely that the current buffers are too
small to fit the decoded frames. In the latter there is the choice to use the
existing buffers or allocate new set of buffer with reduced size. Same applies
to the number of buffers - it can be decreased or increased.

On the OUTPUT queue there is not much to be done. A buffer that contains a
frame with the new resolution will not be dequeued until it is fully processed.

On the CAPTURE queue the application has to be notified about the resolution
change.  The idea proposed during the chat is to introduce a new flag
V4L2_BUF_FLAG_WRONGFORMAT.

1) After all the frames with the old resolution are dequeued a buffer with the
following flags V4L2_BUF_FLAG_ERROR | V4L2_BUF_FLAG_WRONGFORMAT is returned.
2) To acknowledge the resolution change the application should STREAMOFF, check
what has changed and then STREAMON.
3) The application should check with G_FMT how did the format change and the
V4L2_CID_MIN_BUFFERS_FOR_CAPTURE control to check how many buffers are
required.
4) Now it is necessary to resume processing:
  A. If there is no need to change the size of buffers or their number the
application needs only to STREAMON.
  B. If it is necessary to allocate bigger buffers the application should use
CREATE_BUFS to allocate new buffers, the old should be left until the
application is finished and frees them with the DESTROY_BUFS call. S_FMT
should be used to adjust the new format (if necessary and possible in HW).
  C. If only the number of buffers has changed then it is possible to add
buffers with CREATE_BUF or remove spare buffers with DESTROY_BUFS (not yet
implemented to my knowledge).
5) After the application does STREMON the processing should continue. Old
buffers can still be used by the application (as CREATE_BUFS was used), but
they should not be queued (error shall be returned in this case). After the
application is finished with the old buffers it should free them with
DESTROY_BUFS.

I hope I haven't missed anything from our chat. Also please feel free to share
your comments or suggestions.

The log from our chat can be found here
http://www.retiisi.org.uk/v4l2/v4l2codecs-2011-12-01.txt (thanks Sakari).

In a couple of words I would like describe how it is done now - in the MFC
driver that has been posted and accepted in V4L2. Currently the approach is
very similar, but there is no special flag to indicate that the format has
changed. Instead a buffer with no error and bytesused=0 is returned to notify
the application that the resolution/number of buffers has changed. After that
the application does STREAMOFF on the CAPTURE queue, checks the new format with
G_FMT and then reallocates the buffers. To do this the buffers are unmapped and
reqbufs(0) is called. Next step is allocating the new number/size of buffers.
Finally STREAMON is used to resume processing. When this API was implemented
there was no CREATE_BUFS/DESTROY_BUF implementation and plans to implement it.

Handling resolution change with CREATE_BUFS/DESTORY_BUFS and notifying the
application with a dedicated flag seems like a much better solution.

Best wishes,
--
Kamil Debski
Linux Platform Group
Samsung Poland R&D Center



The above message is intended solely for the named addressee and may contain trade secret, industrial technology or privileged and confidential information otherwise protected under applicable law. Any unauthorized dissemination, distribution, copying or use of the information contained in this communication is strictly prohibited. If you have received this communication in error, please notify sender by email and delete this communication immediately.


Powyższa wiadomość przeznaczona jest wyłącznie dla adresata niniejszej wiadomości i może zawierać informacje będące tajemnicą handlową, tajemnicą przedsiębiorstwa oraz informacje o charakterze poufnym chronione obowiązującymi przepisami prawa. Jakiekolwiek nieuprawnione ich rozpowszechnianie, dystrybucja, kopiowanie lub użycie informacji zawartych w powyższej wiadomości jest zabronione. Jeśli otrzymałeś powyższą wiadomość omyłkowo, uprzejmie proszę poinformuj o tym fakcie drogą mailową nadawcę tej wiadomości oraz niezwłocznie usuń powyższą wiadomość ze swojego komputera.
��.n��������+%������w��{.n�����{��g����^n�r������&��z�ޗ�zf���h���~����������_��+v���)ߣ�

[Index of Archives]     [Linux Input]     [Video for Linux]     [Gstreamer Embedded]     [Mplayer Users]     [Linux USB Devel]     [Linux Audio Users]     [Linux Kernel]     [Linux SCSI]     [Yosemite Backpacking]
  Powered by Linux