Re: Questions about Exofs

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]




On Tue, May 15, 2012 at 4:42 PM, Boaz Harrosh <bharrosh@xxxxxxxxxxx> wrote:
> On 05/15/2012 04:09 PM, Idan Kedar wrote:
>
>> On Tue, May 15, 2012 at 3:19 PM, Johannes Schild <JSchild@xxxxxx> wrote:
>>> Hi,
>>>
>>> thanks for your fast reply.
>>>
>>> But now i run into a nullpointer exception while mounting /dev/osd0
>>>
>>>
>>> [  329.919112] Modules linked in: raid456 async_raid6_recov async_pq raid6_pq async_memcpy bnx2i cnic cxgb4i cxgb3i iscsi_tcp be2iscsi iscsi_boot_sysfs uio cxgb4 libcxgbi cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi fuse ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack 8021q ip6table_filter ip6_tables garp stp llc fcoe libfcoe libfc scsi_transport_fc scsi_tgt snd_ens1371 gameport snd_rawmidi snd_ac97_codec ac97_bus joydev snd_seq snd_seq_device snd_pcm ppdev vmw_balloon floppy parport_pc snd_timer snd pcnet32 microcode parport pcspkr mii snd_page_alloc i2c_piix4 i2c_core shpchp uinput mptspi mptscsih mptbase scsi_transport_spi [last unloaded: iscsi_tcp]
>>> [  329.919133]
>>> [  329.919134] Pid: 2234, comm: mount Not tainted 3.3.0-pnfs #1 VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform
>>> [  329.919137] RIP: 0010:[<ffffffff81259799>]  [<ffffffff81259799>] exofs_free_sbi+0x59/0xa0
>>> [  329.919139] RSP: 0018:ffff88003c4e5bc8  EFLAGS: 00010246
>>> [  329.919141] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000001795
>>> [  329.919142] RDX: 0000000000000000 RSI: 0000000000000046 RDI: ffff88003c4f5000
>>> [  329.919143] RBP: ffff88003c4e5bd8 R08: 0000000000000000 R09: 0000000000000000
>>> [  329.919144] R10: 0000000000000000 R11: 0000000000000001 R12: ffff88003c4f5000
>>> [  329.919145] R13: ffff88003c4e5d58 R14: 00000000fffffffb R15: ffff88003bd4e360
>>> [  329.919146] FS:  00007f68ce81d800(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000
>>> [  329.919148] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>>> [  329.919149] CR2: 0000000000000000 CR3: 000000003cbfd000 CR4: 00000000000406f0
>>> [  329.919152] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>> [  329.919155] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>>> [  329.919156] Process mount (pid: 2234, threadinfo ffff88003c4e4000, task ffff88003aa42e40)
>>> [  329.919157] Stack:
>>> [  329.919158]  ffff88003c4f5000 ffff88003c4f5800 ffff88003c4e5d18 ffffffff8125a780
>>> [  329.919160]  ffff88003c4e5c84 0000000081f27618 ffffffff81f27600 ffff88003f80c288
>>> [  329.919162]  ffff88003ddbeee0 0000000000000000 ffff88003c4e5c48 ffffffff812c8433
>>> [  329.919164] Call Trace:
>>> [  329.919166]  [<ffffffff8125a780>] exofs_fill_super+0x440/0x6e0
>>> [  329.919171]  [<ffffffff812c8433>] ? idr_pre_get+0x53/0x90
>>> [  329.919173]  [<ffffffff8125a340>] ? exofs_read_lookup_dev_table+0x480/0x480
>>> [  329.919176]  [<ffffffff81180e18>] mount_nodev+0x58/0xb0
>>> [  329.919178]  [<ffffffff812594ff>] exofs_mount+0x4f/0x80
>>> [  329.919179]  [<ffffffff81181e13>] mount_fs+0x43/0x1b0
>>> [  329.919182]  [<ffffffff8119b56a>] vfs_kern_mount+0x6a/0xf0
>>> [  329.919183]  [<ffffffff8119bd44>] do_kern_mount+0x54/0x110
>>> [  329.919185]  [<ffffffff8119d4b4>] do_mount+0x1a4/0x830
>>> [  329.919188]  [<ffffffff8115da33>] ? alloc_pages_current+0xa3/0x110
>>> [  329.919191]  [<ffffffff811229e4>] ? __get_free_pages+0x14/0x50
>>> [  329.919193]  [<ffffffff8119d18a>] ? copy_mount_options+0x3a/0x170
>>> [  329.919194]  [<ffffffff8119dc80>] sys_mount+0x90/0xe0
>>> [  329.919199]  [<ffffffff81605aa9>] system_call_fastpath+0x16/0x1b
>>> [  329.919200] Code: 66 90 41 8b 84 24 70 03 00 00 83 eb 01 39 c3 72 50 41 8b 94 24 74 03 00 00 01 c2 39 d3 73 42 89 da 29 c2 49 8b 84 24 88 03 00 00 <48> 8b 04 d0 48 8b 38 48 85 ff 74 0c 48 c7 00 00 00 00 00 e8 6f
>>> [  329.919214] RIP  [<ffffffff81259799>] exofs_free_sbi+0x59/0xa0
>>> [  329.919216]  RSP <ffff88003c4e5bc8>
>>> [  329.919217] CR2: 0000000000000000
>>> [  329.919220] ---[ end trace e5a3b1124e42d9e3 ]---
>>>
>>>
>>> Is there any possibility to avoid these failure?
>>>
>>> Just now i am using the 3.3.0 kernel from the linux-pnfs repository.
>>>
>>>
>>> Thanks in adavance
>>>
>>> Cheers
>>> Johannes
>>>
>>> -------- Original-Nachricht --------
>>>> Datum: Tue, 15 May 2012 12:48:08 +0300
>>>> Von: Boaz Harrosh <bharrosh@xxxxxxxxxxx>
>>>> An: Johannes Schild <JSchild@xxxxxx>
>>>> CC: linux-nfs@xxxxxxxxxxxxxxx, open-osd <osd-dev@xxxxxxxxxxxx>
>>>> Betreff: Re: Questions about Exofs
>>>
>>>> On 05/15/2012 12:03 PM, Johannes Schild wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> i try to set up the object-based layout with pnfs.
>>>>> I am using the OSC-OSD as target. It works well and i can login.
>>>>> Trying the scripts "do-*" from git://git.open-osd.org/open-osd.git the
>>>> do-osd script works for me. The do-exofs doesnt, here i have some question:
>>>>>
>>>>> 1. Is the UUID in the do-exofs script optional?
>>>>>
>>>>
>>>>
>>>> No it is not optional
>>>>
>>>>> 2. If no how can i get it for my device?
>>>>>
>>>>
>>>>
>>>> Exactly! the ./do-exofs format command will set this for you, as well as
>>>> mkfs.exofs the
>>>> FS for you.
>>>>
>>>>> 3. It is mandatory to use raid-driver (raid456) in the script? Why?
>>>>>
>>>>
>>>>
>>>> What? where? Rrrr you are right. this is a very old tree. Let me see
>>>> if I have something newer to push. It should all load automatically now.
>>>>
>>>> But yes the dependency on raid456.ko is built in. Though if you use
>>>> raid=0 on the format command line it will not be used in run time.
>>>>
>>>>>
>>>>> The error is:
>>>>> mount -t exofs -o
>>>> osdname=d2683732-c906-4ee1-9dbd-c10c27bb40df,pid=0x10000,_netdev /dev/osd0 /mnt/osd0/
>>>>> mount: wrong fs type, bad option, bad superblock on /dev/osd0,
>>>>>        missing codepage or helper program, or other error
>>>>>        In some cases useful info is found in syslog - try
>>>>>        dmesg | tail  or so
>>>>>
>>>>> dmesg says:
>>>>> exofs: Unable to mount exofs on (null) pid=0x0 err=-22
>>>>>
>>>>
>>>>
>>>> Probably you forgot the "./do-exofs format" stage. Please read the script
>>>> before, you might need to edit it. It is the stage that calls mkfs.exofs
>>>> to make a new filesystem for you. (An OSD is a raw device that can host
>>>> many filesystems each in it's own osd-partition.
>>>>
>>>>>
>>>>> Thanks for answering my questions.
>>>>
>>>>
>>>>> Regards
>>>>> Johannes
>>>>>
>>>>
>>>>
>>>> Cheers
>>>> Boaz
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>> the body of a message to majordomo@xxxxxxxxxxxxxxx
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>> --
>>> NEU: FreePhone 3-fach-Flat mit kostenlosem Smartphone!
>>> Jetzt informieren: http://mobile.1und1.de/?ac=OM.PW.PW003K20328T7073a
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>> the body of a message to majordomo@xxxxxxxxxxxxxxx
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>> Hi,
>> This crash has happened to me as well. I was not able to explicitly
>> resolve it, but I have managed to set up a working pNFS object layout
>> environment. I'll try to give the instructions I use for setting up
>> the MDS, without using the do-* scripts, assuming you managed to set
>> up the data server. (I'm assuming a fedora distro):
>>
>> setup the iscsi initiator:
>>
>> # yum -y install iscsi-initiator-utils
>> # service iscsid start
>> # modprobe osd
>> # iscsiadm -m discovery -t sendtargets -p <your DS> --login
>>
>> on my setup, in some (most) cases, the MDS machine hangs when you
>> power it off. I don't know why this happens.
>>
>> format and mount the OSD device:
>>
>> # modprobe raid456
>
>
> This should not be needed exofs is dependent on ore, dependent on raid456
True, but when setting up the environment my goal was to have a
working environment. If a bug is introduced to this dependence system
and the module wouldn't be loaded, I would have to reluctantly spend
time finding the problem, when I can actually live with such a bug if
the workaround is simply loading the module manually. so as a policy,
I always explicitly load the modules I need when setting up a pNFS
environment.
>
>> # modprobe exofs
>
>> # cd /your/open-osd/project/directory
>> # LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 --dev=/dev/osd0
>>
>
>
> NO you must --format with a UUID at least once.
>
> After the iscsiadm --login stage, there is an osd.ko print in dmesg
> Do you see a print of the form:
>        OSD_NAME               [%s]
> where %s might be empty or it might have something
> If there is something (not []) then maybe it was specified on the
> osd-target (otgtd) command line and all is well. But if not then
> it can never work.
>
>> to me, this works without UUID. I had trouble in setting up exofs with
>> RAID, but if you're not interested in RAID that will do. note that
>> this setup is from the old tree (commit
>> df9cc23ca0e15f2fc7f1d16a21acb87c63038f50), I don't know if and what
>> have been pushed to it recently.
>>
>> note that if you use the --format flag you would have to supply a UUID.
>>
>
>
> You must do that at least once in the lifetime of the device or set
> an osd_name at the otgtd command line.
>
>> now you can mount the object file system. /dev/osd0 is seen as a char
>> device and not a block device, so the system can't read its superblock
>> and tell which file system is it. you have to explicitly specify exofs
>> as the file system, and exofs will do the hard work.:
>
>
> Whaoo I've been using ./do-exofs for ever and did not realize this. I should
> fix that. Thanks It's now on my todo list.
You're welcome :)
>
>>
>> # mount -t exofs -o pid=0x10000 /dev/osd0 /mnt/pnfs
>>
>
>
> Are you sure you have an exofs FS at /mnt/pnfs ? please do an
> # df -h /mnt/pnfs I want to see ?
# df -hT /mnt/pnfs/
Filesystem    Type    Size  Used Avail Use% Mounted on
/dev/osd0    exofs     55G   12G   44G  21% /mnt/pnfs

>
> Does it actually work? OK you know maybe it does. I can see this now.
> If you have a single device it might work, I didn't realize this. I thought
> the mkfs.exofs would not let you.
>
> For sure if you have more then one device (pnfs right?) then it will not
> let you, because the devices have strict order in the device table. Device
> names are not reliable and may change from login to login. You need some
> kind of device-id
Indeed it is a pNFS setup. And indeed I've had trouble when using more
than one OSD.

By the way, several weeks I have tried setting up a RAID 5 environment
with 8 OSDs, 1 mirror and RAID nesting. I then tried cloning and
compiling the kernel tree over this pNFS-OSD-RAID. The result was that
otgtd died, and I don't know why. It didn't dump core anywhere I could
find and the only "log" it has - stdout - didn't give any useful info.
I was going to inquire about this in a couple of weeks when I need to
get this environment working, but since this issue came up, maybe we
can somehow resolve it sooner.
>
>>
>
>
> Sorry guys for the lack of DOCs and proper trees. I neglected
> that for a while now. No one cared.
>
> Let me push some better tested trees.
>
> Thanks
> Boaz

Thanks,
IdanK
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Linux USB Development]     [Linux Media Development]     [Video for Linux]     [Linux NILFS]     [Linux Audio Users]     [Photo]     [Yosemite Info]    [Yosemite Photos]    [POF Sucks]     [Linux Kernel]     [Linux SCSI]     [XFree86]

Add to Google Powered by Linux