On Tue, Mar 31, 2015 at 11:35 AM, Robbie Ko <robbieko@xxxxxxxxxxxx> wrote: > Hi, > > At first, I use fsstress with following script to do random move. > Errors could happen after several rounds of incremental send. > Directory structure generated by fsstress is hard to find simple > reproducible steps, so I started to make a small directory structure > which has several files/directories with the same name. > Then I run the following scripts to randomly move the directory and do > send/receive. Error might occur several rounds later. > > while true; > while true; > do > if [ 1 -eq $((RANDOM%2)) ]; then > src=`find $subvolume_path -type d | sort -R | head -1` > else > src=`find $subvolume_path -type f | sort -R | head -1` > fi > if [ "$src" != "$subvolume_path " ]; then > break > fi > done > while true; > do > dst=`find $subvolume_path -type d | sort -R | head -1` > if [[ "$dst" != "$src"/* ]] && [ "$dst" != "$src" ]; then > break > fi > done > mv $src $dst > done Thanks Robbie. That helps a lot. > > Thanks. > robbieko > > 2015-03-31 17:50 GMT+08:00 Filipe David Manana <fdmanana@xxxxxxxxx>: >> On Tue, Mar 31, 2015 at 10:39 AM, Robbie Ko <robbieko@xxxxxxxxxxxx> wrote: >>> HI, >>> >>> After taking a look at design of send/receive, I feel that severe >>> directory structure movement combined with name collision might be a >>> trouble maker for send to calculate the snapshot difference. >> >> Well the problem is not calculating the differences between snapshots. >> The algorithm of finding new and deleted nodes/leafs in the second >> snapshot, and the new/changed/deleted items is ok. The problem is the >> ability to generate the correct paths (used for renames and other >> operations) at the right moments in time, since the items are >> processed only once and from left to right. >> >>> Therefore, I started using fsstress + random directory movement to >>> test. >> >> Well fsstress alone won't generate these examples you have been >> providing, because it never reuses old names (delete file/dir X, and >> the name X won't ever be reused, rename A to B and name A won't ever >> be used again, etc). So you gotta have something smart there that does >> the directory movement in such a way that preserves names and inverts >> the ancestor-descent relation between directories. >> What is it exactly you are running? >> >>> Since the diff commands might contain a lot of instructions, I >>> try to simplify them before reporting. >> >> Thanks for simplifying it. >> >>> >>> Thanks for your help. >>> >>> Thanks. >>> robbieko >>> >>> 2015-03-31 16:40 GMT+08:00 Filipe David Manana <fdmanana@xxxxxxxxx>: >>>> On Tue, Mar 31, 2015 at 4:06 AM, Robbie Ko <robbieko@xxxxxxxxxxxx> wrote: >>>>> Hi, >>>>> >>>>> I have testing btrfs send/receive recently. >>>>> >>>>> I got an error "send ioctl failed with -12: Cannot allocate memory" on >>>>> send side. >>>>> WARN_ON happened on "len > PATH_MAX" in fs_path_ensure_buf. >>>>> >>>>> I got an error "utime failed: No such file or directory" on receive side. >>>>> The followings are simple reproduced steps and related information. >>>>> >>>>> I have applied the following patch in the url : >>>>> https://patchwork.kernel.org/patch/6113581/. >>>>> However, it did not fix the problem. >>>>> >>>>> Is there any idea about what this might be or how to fix it? >>>>> >>>>> Steps to reproduce: >>>>> >>>>> $ mkfs.btrfs -f /dev/sdb >>>>> $ mount /dev/sdb /mnt >>>>> $ mkfs.btrfs -f /dev/sdc >>>>> $ mount /dev/sdc /mnt2 >>>>> >>>>> $ mkdir -p /mnt/data >>>>> $ mkdir -p /mnt/data/n1/n2 >>>>> $ mkdir -p /mnt/data/n4 >>>>> $ mkdir -p /mnt/data/t6/t7 >>>>> $ mkdir -p /mnt/data/t5 >>>>> $ mkdir -p /mnt/data/t7 >>>>> $ mkdir -p /mnt/data/n4/t2 >>>>> $ mkdir -p /mnt/data/t4 >>>>> $ mkdir -p /mnt/data/t3 >>>>> $ mv /mnt/data/t7 /mnt/data/n4/t2 >>>>> $ mv /mnt/data/t4 /mnt/data/n4/t2/t7 >>>>> $ mv /mnt/data/t5 /mnt/data/n4/t2/t7/t4 >>>>> $ mv /mnt/data/t6 /mnt/data/n4/t2/t7/t4/t5 >>>>> $ mv /mnt/data/n1/n2 /mnt/data/n4/t2/t7/t4/t5/t6 >>>>> $ mv /mnt/data/n1 /mnt/data/n4/t2/t7/t4/t5/t6 >>>>> $ mv /mnt/data/n4/t2/t7/t4/t5/t6/t7 /mnt/data/n4/t2/t7/t4/t5/t6/n2 >>>>> $ mv /mnt/data/t3 /mnt/data/n4/t2/t7/t4/t5/t6/n2/t7 >>>>> >>>>> $ btrfs subvolume snapshot -r /mnt /mnt/snap1 >>>>> >>>>> $ mv /mnt/data/n4/t2/t7/t4/t5/t6/n1 /mnt/data/n4 >>>>> $ mv /mnt/data/n4/t2 /mnt/data/n4/n1 >>>>> $ mv /mnt/data/n4/n1/t2/t7/t4/t5/t6/n2 /mnt/data/n4/n1/t2 >>>>> $ mv /mnt/data/n4/n1/t2/n2/t7/t3 /mnt/data/n4/n1/t2 >>>>> $ mv /mnt/data/n4/n1/t2/t7/t4/t5/t6 /mnt/data/n4/n1/t2 >>>>> $ mv /mnt/data/n4/n1/t2/t7/t4 /mnt/data/n4/n1/t2/t6 >>>>> $ mv /mnt/data/n4/n1/t2/t7 /mnt/data/n4/n1/t2/t3 >>>>> $ mv /mnt/data/n4/n1/t2/n2/t7 /mnt/data/n4/n1/t2 >>>>> >>>>> $ btrfs subvolume snapshot -r /mnt /mnt/snap2 >>>>> >>>>> $ btrfs send /mnt/snap1 | btrfs receive /mnt2 >>>>> $ btrfs send -p /mnt/snap1 /mnt/snap2 | btrfs receive /mnt2 >>>>> ERROR: send ioctl failed with -12: Cannot allocate memory >>>>> ERROR: utimes data/n4/t2/t7/t4/t5/t6/n2 failed. No such file or directory. >>>> >>>> >>>> Hi, >>>> >>>> May I ask if these reproducers come from a real application/workload >>>> or are they generated through some fuzz tester (like fsstress)? >>>> It would be easier, and save time, if you mention how to get such >>>> reproducers instead of sending the same e-mail over and over again but >>>> with different reproducers. >>>> >>>> I'll take a look at it soon. >>>> >>>> thanks >>>> >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >>>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>> >>>> >>>> >>>> -- >>>> Filipe David Manana, >>>> >>>> "Reasonable men adapt themselves to the world. >>>> Unreasonable men adapt the world to themselves. >>>> That's why all progress depends on unreasonable men." >> >> >> >> -- >> Filipe David Manana, >> >> "Reasonable men adapt themselves to the world. >> Unreasonable men adapt the world to themselves. >> That's why all progress depends on unreasonable men." -- Filipe David Manana, "Reasonable men adapt themselves to the world. Unreasonable men adapt the world to themselves. That's why all progress depends on unreasonable men." -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
