[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Using MT9P031 digital sensor



Hi Gary,

On Wednesday 16 November 2011 13:03:11 Gary Thomas wrote:
> On 2011-11-15 18:26, Laurent Pinchart wrote:
> > On Monday 14 November 2011 12:42:54 Gary Thomas wrote:
> >> On 2011-11-11 07:26, Laurent Pinchart wrote:
> >>> On Wednesday 09 November 2011 17:24:26 Gary Thomas wrote:
> >>>> On 2011-11-09 09:18, Laurent Pinchart wrote:
> >>>>> On Wednesday 09 November 2011 12:01:34 Gary Thomas wrote:
> >>>>>> On 2011-11-08 17:54, Laurent Pinchart wrote:
> >>>>>>> On Tuesday 08 November 2011 14:38:55 Gary Thomas wrote:
> >>>>>>>> On 2011-11-08 06:06, Laurent Pinchart wrote:
> >>>>>>>>> On Tuesday 08 November 2011 13:52:25 Gary Thomas wrote:
> >>>>>>>>>> On 2011-11-08 05:30, Javier Martinez Canillas wrote:
> >>>>>>>>>>> On Tue, Nov 8, 2011 at 1:20 PM, Gary Thomas wrote:
> >>>>>>>>>>>> On 2011-11-04 04:37, Laurent Pinchart wrote:
> >>>>>>>>>>>>> On Tuesday 01 November 2011 19:52:49 Gary Thomas wrote:
> >>>>>>>>>>>>>> I'm trying to use the MT9P031 digital sensor with the Media
> >>>>>>>>>>>>>> Controller Framework.  media-ctl tells me that the sensor is
> >>>>>>>>>>>>>> set to capture using SGRBG12  2592x1944
> >>>>>>>>>>>>>> 
> >>>>>>>>>>>>>> Questions:
> >>>>>>>>>>>>>> * What pixel format in ffmpeg does this correspond to?
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> I don't know if ffmpeg supports Bayer formats. The
> >>>>>>>>>>>>> corresponding fourcc in V4L2 is BA12.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> ffmpeg doesn't seem to support these formats
> >>>>>>>>>>>> 
> >>>>>>>>>>>>> If your sensor is hooked up to the OMAP3 ISP, you can then
> >>>>>>>>>>>>> configure the pipeline to include the preview engine and the
> >>>>>>>>>>>>> resizer, and capture YUV data
> >>>>>>>>>>>>> at the resizer output.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> I am using the OMAP3 ISP, but it's a bit unclear to me how to
> >>>>>>>>>>>> set up the pipeline
> >>>>>>>>>>> 
> >>>>>>>>>>> Hi Gary,
> >>>>>>>>>>> 
> >>>>>>>>>>> I'm also using another sensor mtv9034 with OMAP3 ISP, so maybe
> >>>>>>>>>>> I can help you.
> >>>>>>>>>>> 
> >>>>>>>>>>>> using media-ctl (I looked for documentation on this tool, but
> >>>>>>>>>>>> came up dry - is there any?)
> >>>>>>>>>>>> 
> >>>>>>>>>>>> Do you have an example of how to configure this using the
> >>>>>>>>>>>> OMAP3 ISP?
> >>>>>>>>>>> 
> >>>>>>>>>>> This is how I configure the pipeline to connect the CCDC with
> >>>>>>>>>>> the Previewer and Resizer:
> >>>>>>>>>>> 
> >>>>>>>>>>> ./media-ctl -l '"mt9v032 3-005c":0->"OMAP3 ISP CCDC":0[1]'
> >>>>>>>>>>> ./media-ctl -l '"OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1]'
> >>>>>>>>>>> ./media-ctl -l '"OMAP3 ISP preview":1->"OMAP3 ISP
> >>>>>>>>>>> resizer":0[1]' ./media-ctl -l '"OMAP3 ISP resizer":1->"OMAP3
> >>>>>>>>>>> ISP resizer output":0[1]' ./media-ctl -f '"mt9v032
> >>>>>>>>>>> 3-005c":0[SGRBG10 752x480]' ./media-ctl -f  '"OMAP3 ISP
> >>>>>>>>>>> CCDC":0 [SGRBG10 752x480]' ./media-ctl -f  '"OMAP3 ISP CCDC":1
> >>>>>>>>>>> [SGRBG10 752x480]' ./media-ctl -f  '"OMAP3 ISP preview":0
> >>>>>>>>>>> [SGRBG10 752x479]' ./media-ctl -f  '"OMAP3 ISP resizer":0
> >>>>>>>>>>> [YUYV 734x471]' ./media-ctl -f  '"OMAP3 ISP resizer":1 [YUYV
> >>>>>>>>>>> 640x480]'
> >>>>>>>>>>> 
> >>>>>>>>>>> Hope it helps,
> >>>>>>>>>> 
> >>>>>>>>>> Thanks, I'll give this a try.
> >>>>>>>>>> 
> >>>>>>>>>> I assume that your sensor is probably larger than 752x480 (the
> >>>>>>>>>> mt9p031 is 2592x1944 raw) and that setting the smaller frame
> >>>>>>>>>> size enables some scaling and/or cropping in the driver?
> >>>>>>>>> 
> >>>>>>>>> The mt9v034 is a wide VGA 752x480 sensor if I'm not mistaken. You
> >>>>>>>>> should modify the resolutions in the above commands according to
> >>>>>>>>> your sensor. Note that the CCDC crops online line when outputting
> >>>>>>>>> data to the preview engine, and that the preview engine crops 18
> >>>>>>>>> columsn and 8 lines. You can then scale the image by modifying
> >>>>>>>>> the resizer output size.
> >>>>>>>> 
> >>>>>>>> Thanks.  After much trial and error (and some kernel printks to
> >>>>>>>> 
> >>>>>>>> understand what parameters were failing), I came up with this
> > 
> > sequence:
> >>>>>>>>        media-ctl -r
> >>>>>>>>        media-ctl -l '"mt9p031 3-005d":0->"OMAP3 ISP CCDC":0[1]'
> >>>>>>>>        media-ctl -l '"OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1]'
> >>>>>>>>        media-ctl -l '"OMAP3 ISP preview":1->"OMAP3 ISP
> >>>>>>>>        resizer":0[1]' media-ctl -l '"OMAP3 ISP resizer":1->"OMAP3
> >>>>>>>>        ISP resizer output":0[1]' media-ctl -f '"mt9p031
> >>>>>>>>        3-005d":0[SGRBG12 2592x1944]' media-ctl -f  '"OMAP3 ISP
> >>>>>>>>        CCDC":0 [SGRBG12 2592x1944]'
> >>>>>>>>        media-ctl -f  '"OMAP3 ISP CCDC":1 [SGRBG12 2592x1944]'
> >>>>>>>>        media-ctl -f  '"OMAP3 ISP preview":0 [SGRBG12 2592x1943]'
> >>>>>>>>        media-ctl -f  '"OMAP3 ISP resizer":0 [YUYV 2574x1935]'
> >>>>>>>>        media-ctl -f  '"OMAP3 ISP resizer":1 [YUYV 642x483]'
> >>>>>>>> 
> >>>>>>>> When I tried to grab though, I got this:
> >>>>>>>> 
> >>>>>>>> # yavta --capture=4 -f YUYV -s 642x483 -F /dev/video6
> >>>>>>>> Device /dev/video6 opened.
> >>>>>>>> Device `OMAP3 ISP resizer output' on `media' is a video capture
> >>>>>>>> device. Video format set: YUYV (56595559) 642x483 buffer size
> >>>>>>>> 633696 Video format: YUYV (56595559) 642x483 buffer size 633696
> >>>>>>>> 8 buffers requested.
> >>>>>>>> length: 633696 offset: 0
> >>>>>>>> Buffer 0 mapped at address 0x4028c000.
> >>>>>>>> length: 633696 offset: 634880
> >>>>>>>> Buffer 1 mapped at address 0x403d0000.
> >>>>>>>> length: 633696 offset: 1269760
> >>>>>>>> Buffer 2 mapped at address 0x404b3000.
> >>>>>>>> length: 633696 offset: 1904640
> >>>>>>>> Buffer 3 mapped at address 0x4062b000.
> >>>>>>>> length: 633696 offset: 2539520
> >>>>>>>> Buffer 4 mapped at address 0x406d6000.
> >>>>>>>> length: 633696 offset: 3174400
> >>>>>>>> Buffer 5 mapped at address 0x40821000.
> >>>>>>>> length: 633696 offset: 3809280
> >>>>>>>> Buffer 6 mapped at address 0x4097c000.
> >>>>>>>> length: 633696 offset: 4444160
> >>>>>>>> Buffer 7 mapped at address 0x40adf000.
> >>>>>>>> 
> >>>>>>>> Unable to handle kernel NULL pointer dereference at virtual
> >>>>>>>> address 00000018
> >>>>>>> 
> >>>>>>> Ouch :-(
> >>>>>>> 
> >>>>>>> Could you please verify that arch/arm/mach-omap2/board-overo.c
> >>>>>>> includes the following code, and that CONFIG_OMAP_MUX is enabled ?
> >>>>>> 
> >>>>>> I'm not using an Overo board - rather one of our own internal
> >>>>>> designs.
> >>>>> 
> >>>>> My bad, sorry.
> >>>>> 
> >>>>>> I have verified that the pull up/down on those pins is disabled.
> >>>>>> 
> >>>>>> The failure is coming from this code in ispccdc.c
> >>>>>> 
> >>>>>>       static void ccdc_hs_vs_isr(struct isp_ccdc_device *ccdc)
> >>>>>>       {
> >>>>>> 	  
> >>>>>> 	  struct isp_pipeline *pipe =
> >>>>>> 		
> >>>>>> 		to_isp_pipeline(&ccdc->video_out.video.entity);
> >>>>>> 
> >>>>>> The value of pipe is NULL which leads to the failure.
> >>>>>> 
> >>>>>> Questions:
> >>>>>> * I assume that getting HS/VS interrupts is correct in this mode?
> >>>>>> * Why is the statement not written (as all others are)
> >>>>>> 
> >>>>>> 	struct isp_pipeline *pipe = to_isp_pipeline(&ccdc->subdev.entity);
> >>>>>> 	
> >>>>>>       I tried this change and the kernel doesn't crash.
> >>>>>> 
> >>>>>> I've found that I can get raw frames out of CCDC, but I don't get
> >>>>>> anything at all when the output continues through the preview and/or
> >>>>>> resize nodes.
> >>>>>> 
> >>>>>> Ideas?
> >>>>> 
> >>>>> I'm really puzzled, this should have been caught much earlier :-)
> >>>>> 
> >>>>> Your analysis makes sense. Would you like to submit a patch yourself
> >>>>> ? If not I can do it.
> >>>> 
> >>>> Sure, I can submit a patch. I would like to figure out why it's not
> >>>> working first.
> >>> 
> >>> Oops, I've overlooked that, sorry.
> >>> 
> >>>> Any ideas how I can debug this? I can't seem to get anything past the
> >>>> CCDC, e.g. into the preview or resize units. Is there some way to
> >>>> trace packets/data through the various stages? Any ideas what might
> >>>> cause it to stall?
> >>> 
> >>> How have you configured your pipeline ? Can you try tracing the preview
> >>> engine and/or resizer interrupts ?
> >> 
> >> Here's my pipeline:
> >>     media-ctl -r
> >>     media-ctl -l '"mt9p031 3-005d":0->"OMAP3 ISP CCDC":0[1]'
> >>     media-ctl -l '"OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1]'
> >>     media-ctl -l '"OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1]'
> >>     media-ctl -l '"OMAP3 ISP resizer":1->"OMAP3 ISP resizer
> >>     output":0[1]' media-ctl -f '"mt9p031 3-005d":0[SGRBG12 2592x1944]'
> >>     media-ctl -f  '"OMAP3 ISP CCDC":0 [SGRBG10 2592x1944]'
> >>     media-ctl -f  '"OMAP3 ISP CCDC":1 [SGRBG10 2592x1944]'
> >>     media-ctl -f  '"OMAP3 ISP preview":0 [SGRBG10 2592x1943]'
> >>     media-ctl -f  '"OMAP3 ISP resizer":0 [YUYV 2574x1935]'
> >>     media-ctl -f  '"OMAP3 ISP resizer":1 [YUYV 642x483]'
> >> 
> >> The full media-ctl dump is at http://www.mlbassoc.com/misc/pipeline.out
> >> 
> >> When I try to grab from /dev/video6 (output node of resizer), I see
> >> only previewer interrupts, no resizer interrrupts.  I added a simple
> >> printk at each of the previewer/resizer *_isr functions, and I only
> >> 
> >> ever see this one:
> >>     omap3isp_preview_isr_frame_sync.1373
> >> 
> >> Can you give me an overview of what events/interrupts should occur so
> >> I can try to trace through the ISP to see where it is failing?
> > 
> > The CCDC generates VD0, VD1 and HS/VS interrupts regardless of whether it
> > processes video or not, as long as it receives a video stream at its
> > input. The preview engine and resizer will only generate an interrupt
> > after writing an image to memory. With your above configuration VD0,
> > VD1, HS/VS and resizer interrupts should be generated.
> > 
> > Your pipeline configuration looks correct, except that the downscaling
> > factor is slightly larger than 4. Could you try to setup the resizer to
> > output a 2574x1935 image instead of 642x483 ? If that works, try to
> > downscale to 660x496. If that works as well, the driver should be fixed
> > to disallow resolutions that won't work.
> 
> No change.  I also tried using only the previewer like this:
>    media-ctl -r
>    media-ctl -l '"mt9p031 3-005d":0->"OMAP3 ISP CCDC":0[1]'
>    media-ctl -l '"OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1]'
>    media-ctl -l '"OMAP3 ISP preview":1->"OMAP3 ISP preview output":0[1]'
>    media-ctl -f '"mt9p031 3-005d":0[SGRBG12 2592x1944]'
>    media-ctl -f  '"OMAP3 ISP CCDC":0 [SGRBG12 2592x1944]'
>    media-ctl -f  '"OMAP3 ISP CCDC":1 [SGRBG10 2592x1944]'
>    media-ctl -f  '"OMAP3 ISP preview":0 [SGRBG10 2592x1943]'
>    media-ctl -f  '"OMAP3 ISP preview":1 [YUYV 2574x1935]'
> 
>    yavta --capture=4 -f YUYV -s 2574x1935 -F /dev/video4
> 
> I still only get the frame sync interrupts in the previewer, no buffer
> interrupts, hence no data flowing to my application.  What else can I
> look at?

Do you get VD0 and VD1 interrupts ?

-- 
Regards,

Laurent Pinchart
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Linux Input]     [Video for Linux]     [Mplayer Users]     [Linux USB Devel]     [Linux Audio Users]     [Photos]     [Yosemite Photos]     [Linux Kernel]     [Linux SCSI]     [XFree86]     [Devices]     [Yosemite Backpacking]

Add to Google Powered by Linux