[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [RFC] media DT bindings



Hi Sylwester

Thanks for your comments.

On Fri, 13 Jul 2012, Sylwester Nawrocki wrote:

> Hi,
> 
> Cc: devicetree-disscuss@xxxxxxxxxxxxxxxx
> 
> On 07/11/2012 04:27 PM, Guennadi Liakhovetski wrote:
> > Hi all
> > 
> > Background
> > ==========
> > 
> > With ARM adoption of flat Device Trees a need arises to move platform
> > device descriptions and their data from platform files to DT. This has
> > also to be done for media devices, e.g., video capture and output
> > interfaces, data processing devices, camera sensors, TV decoders and
> > encoders. This RFC is trying to spawn a discussion to define standard V4L
> > DT bindings. The first version will concentrate on the capture path,
> > mostly taking care of simple capture-interface - camera sensor / TV
> > decoder configurations. Since the author is not working intensively yet
> > with the Media Controller API, pad-level configuration, these topics might
> > be underrepresented in this RFC. I hope others, actively working in these
> > areas, will fill me in on them.
> 
> We've done some work on device tree support for SoC camera interface
> with driver based on the Media Controller API, I've posted some RFC
> patches a few weeks ago:
> http://www.mail-archive.com/devicetree-discuss@xxxxxxxxxxxxxxxx/msg14769.html
> But unfortunately didn't receive any comments,

You have now ;-)

> perhaps because the actual
> bindings were not abstracted enough from a specific hardware support.
> An updated version of these patch set can be found here:
> https://github.com/snawrocki/linux/commits/camera-of-2
> 
> Of course we shouldn't be forgetting that underlying bindings need to 
> be the same, regardless of the drivers are based on soc_camera, Media 
> Controller/subdev pad-level frameworks, or something else.

Of course, the more properties are common - the better. I also made sure 
not to introduce any soc-camera specific properties. But since I mostly 
work with those systems, I am not fully aware of requirements, imposed by 
other hardware types, so, I hope others will contribute to this work:-)

> Anything linux specific in the bindings would be inappropriate.

Not sure what you mean here - which Linux specific bindings and why they 
wouldn't be appropriate? Don't think our bindings would be used by any 
other OS kernels.

> > Overview
> > ========
> > 
> > As mentioned above, typical configurations, that we'll be dealing with
> > consist of a DMA data capture engine, one or more data sources like camera
> > sensors, possibly some data processing units. Data capture and processing
> > engines are usually platform devices, whereas data source devices are
> > typically I2C slaves. Apart from defining each device we'll also describe
> > connections between them as well as properties of those connections.
> > 
> > Capture devices
> > ==============================
> > 
> > These are usually platform devices, integrated into respective SoCs. There
> > also exist external image processing devices, but they are rare. Obvious
> > differences between them and integrated devices include a different bus
> > attribution and a need to explicitly describe the connection to the SoC.
> > As far as capture devices are concerned, their configuration will
> > typically include a few device-specific bindings, as well as standard
> > ones. Standard bindings will include the usual "reg," "interrupts,"
> > "clock-frequency" properties.
> > 
> > It is more complex to describe external links. We need to describe
> > configurations, used with various devices, attached to various pads. It is
> > proposed to describe such links as child nodes. Each such link will
> > reference a client pad, a local pad and specify the bus configuration. The
> > media bus can be either parallel or serial, e.g., MIPI CSI-2. It is
> > proposed to describe both the bus-width in the parallel case and the
> > number of lanes in the serial case, using the standard "bus-width"
> > property.
> > 
> > On the parallel bus common properties include signal polarities, possibly
> > data line shift (8 if lines 15:8 are used, 2 if 9:2, and 0 if lines 7:0),
> > protocol (e.g., BT.656). Additionally device-specific properties can be
> > defined.
> > 
> > A MIPI CSI-2 bus common properties would include, apart from the number of
> > lanes, routed to that client, the clock frequency, a channel number,
> > possibly CRC and ECC flags.
> > 
> > An sh-mobile CEU DT node could look like
> > 
> > 	ceu0@0xfe910000 = {
> > 		compatible = "renesas,sh-mobile-ceu";
> > 		reg =<0xfe910000 0xa0>;
> > 		interrupts =<0x880>;
> > 		bus-width =<16>;		/* #lines routed on the board */
> > 		clock-frequency =<50000000>;	/* max clock */
> > 		#address-cells =<1>;
> > 		#size-cells =<0>;
> > 		...
> > 		ov772x-1 = {
> > 			reg =<0>;
> > 			client =<&ov772x@0x21-0>;
> > 			local-pad = "parallel-sink";
> > 			remote-pad = "parallel-source";
> 
> I'm not sure I like that. Is it really needed when we already have
> the child/parent properties around ?

I think it is. Both the host and the client can have multiple pads (e.g., 
parallel / serial). These properties specify which pads are used and make 
the translation between DT data and our subdev / pad APIs simpler.

> > 			bus-width =<8>;	/* used data lines */
> > 			data-shift =<0>;	/* lines 7:0 are used */
> > 			hsync-active =<1>;	/* active high */
> > 			vsync-active =<1>;	/* active high */
> 
> In the end I took a bit different approach, similar to how the interrupt 
> flag bindings are defined:
> https://github.com/snawrocki/linux/commit/c17a61a07008eeb8faea0205f7cc440545641adb
> 
> However using a separate boolean for each signal, as you proposed, might 
> not be that much of string parsing, and it would have hurt less if this 
> would have been done is some common helper modules.

Personally, I think, I would prefer to have 1 property per flag. Whether 
we use an integer as above or a boolean as in this patch:

http://thread.gmane.org/gmane.linux.drivers.devicetree/17495

can be discussed.

> > 			pclk-sample =<1>;	/* rising */
> > 			clock-frequency =<24000000>;
> > 		};
> > 	};
> > 
> > Client devices
> > ==============
> > 
> > Client nodes are children on their respective busses, e.g., i2c. This
> > placement leads to these devices being possibly probed before respective
> > host interfaces, which will fail due to known reasons. Therefore client
> > drivers have to be adapted to request a delayed probing, as long as the
> > respective video host hasn't probed.
> 
> I doubt this is going to be sufficient. Video bridge drivers may require
> all their sub-devices (e.g. I2C client devices) to be registered, in order
> to complete their probing. This was discussed in the past:
> http://www.mail-archive.com/devicetree-discuss@xxxxxxxxxxxxxxxx/msg06595.html

How about this:

 - if sensors get probed before the host, they request deferred probing;
 - when the host gets probed, it checks, which clients it should be 
   getting, enables respective clocks, registers a notifier on respective 
   busses (&i2c_bus_type for I2C) and returns success
 - when after this sensors are re-probed, the notifier gets called and the 
   host can complete its initialisation

> So we ended up with defining an aggregate DT node that happens to be
> associated with the camera media device driver.
> 
> Here is an example of how it might look like:
> https://github.com/snawrocki/linux/commit/eae639132681df6c068dc869bb8973f8d9d3efa1

Yeah, it can be done if absolutely needed, but I wouldn't make it 
mandatory. On simpler sistems 1 host and 1 sensor nodes should suffice.

> > Client nodes will include all the properties, usual for their busses.
> > Additionally they will specify properties private to this device type and
> > common for all V4L2 client devices - device global and per-link. I think,
> 
> Sounds good.
> 
> > we should make it possible to define client devices, that can at run-time
> > be connected to different sinks, even though such configurations might not
> > be very frequent. To achieve this we also specify link information in
> > child devices, similar to those in host nodes above. This also helps
> > uniformity and will let us implement and use a universal link-binding
> > parser. So, a node, that has been referenced above could look like
> 
> It seems dubious to me to push media links' description into DT. Links can 
> be configurable, and I don't think it's a rare situation. I'm inclined to
> code the interconnections within a composite device driver.

Shouldn't a list of all possible links be provided by the platform? If a 
certain pad can participate in several links, all of them should be 
specified and at run-time we just switch between them?

> > 	ov772x@0x21-0 = {
> > 		compatible = "omnivision,ov772x";
> > 		reg =<0x21>;
> > 		vdd-supply =<&regulator>;
> > 		bus-width =<10>;
> > 		#address-cells =<1>;
> > 		#size-cells =<0>;
> > 		...
> > 		ceu0-1 = {
> > 			reg =<0>;
> > 			media-parent =<&ceu0@0xfe910000>;
> > 			bus-width =<8>;
> > 			hsync-active =<1>;
> > 			vsync-active =<0>;	/* who came up with an inverter here?... */
> > 			pclk-sample =<1>;
> > 		};
> 
> Are these mostly supposed to be properties specific to a sub-device and
> used by a host ?

These parameters specify the subdevice configuration and are also used by 
the subdevice to configure its signal polarities.

> If so, how about adding them under the host or the aggregate node grouped
> into a sub-device specific child node ?

We need 2 copies of them - for both pads, that's exactly the reason for 
the above inverter "joke."

> > 	};
> > 
> > Data processors
> > ===============
> > 
> > Data processing modules include resizers, codecs, rotators, serialisers,
> > etc. A node for an sh-mobile CSI-2 subdevice could look like
> > 
> > 	csi2@0xffc90000 = {
> > 		compatible = "renesas,sh-mobile-csi2";
> > 		reg =<0xffc90000 0x1000>;
> > 		interrupts =<0x17a0>;
> > 		bus-width =<4>;
> > 		clock-frequency =<30000000>;
> > 		...
> > 		imx074-1 = {
> > 			client =<&imx074@0x1a-0>;
> > 			local-pad = "csi2-sink";
> > 			remote-pad = "csi2-source";
> > 			bus-width =<2>;
> > 			clock-frequency =<25000000>;
> > 			csi2-crc;
> > 			csi2-ecc;
> > 			sh-csi2,phy =<0>;
> > 		};
> > 		ceu0 = {
> > 			media-parent =<&ceu0@0xfe910000>;
> > 			immutable;
> 
> > 		};
> > 	};
> > 
> > The respective child binding in the CEU node could then look like
> > 
> > 		csi2-1 = {
> > 			reg =<1>;
> > 			client =<&csi2@0xffc90000>;
> > 			immutable;
> > 		};
> 
> It look a bit complex, and could more complex when we run into a system
> supporting more complex interconnections. I would prefer some sort of
> more flat structure...

Ideas welcome:-)

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Linux Input]     [Video for Linux]     [Mplayer Users]     [Linux USB Devel]     [Linux Audio Users]     [Photos]     [Yosemite Photos]     [Linux Kernel]     [Linux SCSI]     [XFree86]     [Devices]     [Yosemite Backpacking]

Add to Google Powered by Linux