B.H. On Tue, Jul 7, 2015 at 4:14 PM, Mordechay Kaganer <mkaganer@xxxxxxxxx> wrote: > > > The conclusion is: to actually reclaim the duplicated space you have > to include all snapshots that may point to the file. > Tried to dedupe the real data, including all snapshots. Still no free space gain. This time, this looks like dedupe didn't actually work. The log: # duperemove: Showing 12 identical extents with id 572cbc6e Start Length Filename 0.0 54.9M "/test_btrfs/snapshots/2015-06-13/subvol1/dir770/21+/3065.wav" 0.0 54.9M "/test_btrfs/snapshots/2015-06-13/subvol1/dir770.sav/21/3065.wav" 0.0 54.9M "/test_btrfs/snapshots/2015-06-16/subvol1/dir770/21+/3065.wav" 0.0 54.9M "/test_btrfs/snapshots/2015-06-16/subvol1/dir770.sav/21/3065.wav" 0.0 54.9M "/test_btrfs/snapshots/2015-06-20/subvol1/dir770/21+/3065.wav" 0.0 54.9M "/test_btrfs/snapshots/2015-06-20/subvol1/dir770.sav/21/3065.wav" 0.0 54.9M "/test_btrfs/snapshots/2015-06-23/subvol1/dir770/21+/3065.wav" 0.0 54.9M "/test_btrfs/snapshots/2015-06-23/subvol1/dir770.sav/21/3065.wav" 0.0 54.9M "/test_btrfs/snapshots/2015-07-05/subvol1/dir770/21+/3065.wav" 0.0 54.9M "/test_btrfs/snapshots/2015-07-05/subvol1/dir770.sav/21/3065.wav" 0.0 54.9M "/test_btrfs/subvol1/dir770/21+/3065.wav" 0.0 54.9M "/test_btrfs/subvol1/dir770.sav/21/3065.wav" .... [0x9b7800] Try to dedupe extents with id 572cbc6e [0x9b7800] Dedupe 1 extents (id: 572cbc6e) with target: (0.0, 54.9M), "/test_btrfs/snapshots/2015-06-13/subvol1/dir770/21+/3065.wav" .... Kernel processed data (excludes target files): 49.8G Comparison of extent info shows a net change in shared extents of: 0.0 # show-shared-extents $file | head -5: show-shared-extents /test_btrfs/snapshots/2015-06-13/subvol1/dir770/21+/3065.wav: (fiemap) [0] fe_logical: 0, fe_length: 131072, fe_physical: 8178629230592, fe_flags: 0x2008 (encoded shared ) (fiemap) [1] fe_logical: 131072, fe_length: 131072, fe_physical: 8178629300224, fe_flags: 0x2008 (encoded shared ) (fiemap) [2] fe_logical: 262144, fe_length: 131072, fe_physical: 8178629369856, fe_flags: 0x2008 (encoded shared ) (fiemap) [3] fe_logical: 393216, fe_length: 131072, fe_physical: 8178629451776, fe_flags: 0x2008 (encoded shared ) (fiemap) [4] fe_logical: 524288, fe_length: 131072, fe_physical: 8178629533696, fe_flags: 0x2008 (encoded shared ) show-shared-extents /test_btrfs/snapshots/2015-06-13/subvol1/dir770.sav/21/3065.wav: (fiemap) [0] fe_logical: 0, fe_length: 131072, fe_physical: 8178629230592, fe_flags: 0x2008 (encoded shared ) (fiemap) [1] fe_logical: 131072, fe_length: 131072, fe_physical: 8178629300224, fe_flags: 0x2008 (encoded shared ) (fiemap) [2] fe_logical: 262144, fe_length: 131072, fe_physical: 8178629369856, fe_flags: 0x2008 (encoded shared ) (fiemap) [3] fe_logical: 393216, fe_length: 131072, fe_physical: 8178629451776, fe_flags: 0x2008 (encoded shared ) (fiemap) [4] fe_logical: 524288, fe_length: 131072, fe_physical: 8178629533696, fe_flags: 0x2008 (encoded shared ) show-shared-extents /test_btrfs/snapshots/2015-06-16/subvol1/dir770/21+/3065.wav: (fiemap) [0] fe_logical: 0, fe_length: 131072, fe_physical: 8178629230592, fe_flags: 0x2008 (encoded shared ) (fiemap) [1] fe_logical: 131072, fe_length: 131072, fe_physical: 8178629300224, fe_flags: 0x2008 (encoded shared ) (fiemap) [2] fe_logical: 262144, fe_length: 131072, fe_physical: 8178629369856, fe_flags: 0x2008 (encoded shared ) (fiemap) [3] fe_logical: 393216, fe_length: 131072, fe_physical: 8178629451776, fe_flags: 0x2008 (encoded shared ) (fiemap) [4] fe_logical: 524288, fe_length: 131072, fe_physical: 8178629533696, fe_flags: 0x2008 (encoded shared ) show-shared-extents /test_btrfs/snapshots/2015-06-16/subvol1/dir770.sav/21/3065.wav: (fiemap) [0] fe_logical: 0, fe_length: 131072, fe_physical: 7960106323968, fe_flags: 0x2008 (encoded shared ) (fiemap) [1] fe_logical: 131072, fe_length: 131072, fe_physical: 7960106393600, fe_flags: 0x2008 (encoded shared ) (fiemap) [2] fe_logical: 262144, fe_length: 131072, fe_physical: 7960106463232, fe_flags: 0x2008 (encoded shared ) (fiemap) [3] fe_logical: 393216, fe_length: 131072, fe_physical: 7960106545152, fe_flags: 0x2008 (encoded shared ) (fiemap) [4] fe_logical: 524288, fe_length: 131072, fe_physical: 7960106627072, fe_flags: 0x2008 (encoded shared ) show-shared-extents /test_btrfs/snapshots/2015-06-20/subvol1/dir770/21+/3065.wav: (fiemap) [0] fe_logical: 0, fe_length: 131072, fe_physical: 8178629230592, fe_flags: 0x2008 (encoded shared ) (fiemap) [1] fe_logical: 131072, fe_length: 131072, fe_physical: 8178629300224, fe_flags: 0x2008 (encoded shared ) (fiemap) [2] fe_logical: 262144, fe_length: 131072, fe_physical: 8178629369856, fe_flags: 0x2008 (encoded shared ) (fiemap) [3] fe_logical: 393216, fe_length: 131072, fe_physical: 8178629451776, fe_flags: 0x2008 (encoded shared ) (fiemap) [4] fe_logical: 524288, fe_length: 131072, fe_physical: 8178629533696, fe_flags: 0x2008 (encoded shared ) show-shared-extents /test_btrfs/snapshots/2015-06-20/subvol1/dir770.sav/21/3065.wav: (fiemap) [0] fe_logical: 0, fe_length: 131072, fe_physical: 7960106323968, fe_flags: 0x2008 (encoded shared ) (fiemap) [1] fe_logical: 131072, fe_length: 131072, fe_physical: 7960106393600, fe_flags: 0x2008 (encoded shared ) (fiemap) [2] fe_logical: 262144, fe_length: 131072, fe_physical: 7960106463232, fe_flags: 0x2008 (encoded shared ) (fiemap) [3] fe_logical: 393216, fe_length: 131072, fe_physical: 7960106545152, fe_flags: 0x2008 (encoded shared ) (fiemap) [4] fe_logical: 524288, fe_length: 131072, fe_physical: 7960106627072, fe_flags: 0x2008 (encoded shared ) show-shared-extents /test_btrfs/snapshots/2015-06-23/subvol1/dir770/21+/3065.wav: (fiemap) [0] fe_logical: 0, fe_length: 131072, fe_physical: 8178629230592, fe_flags: 0x2008 (encoded shared ) (fiemap) [1] fe_logical: 131072, fe_length: 131072, fe_physical: 8178629300224, fe_flags: 0x2008 (encoded shared ) (fiemap) [2] fe_logical: 262144, fe_length: 131072, fe_physical: 8178629369856, fe_flags: 0x2008 (encoded shared ) (fiemap) [3] fe_logical: 393216, fe_length: 131072, fe_physical: 8178629451776, fe_flags: 0x2008 (encoded shared ) (fiemap) [4] fe_logical: 524288, fe_length: 131072, fe_physical: 8178629533696, fe_flags: 0x2008 (encoded shared ) show-shared-extents /test_btrfs/snapshots/2015-06-23/subvol1/dir770.sav/21/3065.wav: (fiemap) [0] fe_logical: 0, fe_length: 131072, fe_physical: 7960106323968, fe_flags: 0x2008 (encoded shared ) (fiemap) [1] fe_logical: 131072, fe_length: 131072, fe_physical: 7960106393600, fe_flags: 0x2008 (encoded shared ) (fiemap) [2] fe_logical: 262144, fe_length: 131072, fe_physical: 7960106463232, fe_flags: 0x2008 (encoded shared ) (fiemap) [3] fe_logical: 393216, fe_length: 131072, fe_physical: 7960106545152, fe_flags: 0x2008 (encoded shared ) (fiemap) [4] fe_logical: 524288, fe_length: 131072, fe_physical: 7960106627072, fe_flags: 0x2008 (encoded shared ) show-shared-extents /test_btrfs/snapshots/2015-07-05/subvol1/dir770/21+/3065.wav: (fiemap) [0] fe_logical: 0, fe_length: 131072, fe_physical: 8178629230592, fe_flags: 0x2008 (encoded shared ) (fiemap) [1] fe_logical: 131072, fe_length: 131072, fe_physical: 8178629300224, fe_flags: 0x2008 (encoded shared ) (fiemap) [2] fe_logical: 262144, fe_length: 131072, fe_physical: 8178629369856, fe_flags: 0x2008 (encoded shared ) (fiemap) [3] fe_logical: 393216, fe_length: 131072, fe_physical: 8178629451776, fe_flags: 0x2008 (encoded shared ) (fiemap) [4] fe_logical: 524288, fe_length: 131072, fe_physical: 8178629533696, fe_flags: 0x2008 (encoded shared ) show-shared-extents /test_btrfs/snapshots/2015-07-05/subvol1/dir770.sav/21/3065.wav: (fiemap) [0] fe_logical: 0, fe_length: 131072, fe_physical: 7960106323968, fe_flags: 0x2008 (encoded shared ) (fiemap) [1] fe_logical: 131072, fe_length: 131072, fe_physical: 7960106393600, fe_flags: 0x2008 (encoded shared ) (fiemap) [2] fe_logical: 262144, fe_length: 131072, fe_physical: 7960106463232, fe_flags: 0x2008 (encoded shared ) (fiemap) [3] fe_logical: 393216, fe_length: 131072, fe_physical: 7960106545152, fe_flags: 0x2008 (encoded shared ) (fiemap) [4] fe_logical: 524288, fe_length: 131072, fe_physical: 7960106627072, fe_flags: 0x2008 (encoded shared ) show-shared-extents /test_btrfs/subvol1/dir770/21+/3065.wav: (fiemap) [0] fe_logical: 0, fe_length: 131072, fe_physical: 8178629230592, fe_flags: 0x2008 (encoded shared ) (fiemap) [1] fe_logical: 131072, fe_length: 131072, fe_physical: 8178629300224, fe_flags: 0x2008 (encoded shared ) (fiemap) [2] fe_logical: 262144, fe_length: 131072, fe_physical: 8178629369856, fe_flags: 0x2008 (encoded shared ) (fiemap) [3] fe_logical: 393216, fe_length: 131072, fe_physical: 8178629451776, fe_flags: 0x2008 (encoded shared ) (fiemap) [4] fe_logical: 524288, fe_length: 131072, fe_physical: 8178629533696, fe_flags: 0x2008 (encoded shared ) show-shared-extents /test_btrfs/subvol1/dir770.sav/21/3065.wav: (fiemap) [0] fe_logical: 0, fe_length: 131072, fe_physical: 7960106323968, fe_flags: 0x2008 (encoded shared ) (fiemap) [1] fe_logical: 131072, fe_length: 131072, fe_physical: 7960106393600, fe_flags: 0x2008 (encoded shared ) (fiemap) [2] fe_logical: 262144, fe_length: 131072, fe_physical: 7960106463232, fe_flags: 0x2008 (encoded shared ) (fiemap) [3] fe_logical: 393216, fe_length: 131072, fe_physical: 7960106545152, fe_flags: 0x2008 (encoded shared ) (fiemap) [4] fe_logical: 524288, fe_length: 131072, fe_physical: 7960106627072, fe_flags: 0x2008 (encoded shared ) Note that here we actually have 2 duplicate files (subvol1/dir770/21+/3065.wav, subvol1/dir770.sav/21/3065.wav) and 5 snapshots of each of file. We attempted to dedupe all 12 instances at once but according to fe_physical we still have 2 copies of the data. Why is this happening? -- משיח NOW! Moshiach is coming very soon, prepare yourself! יחי אדוננו מורינו ורבינו מלך המשיח לעולם ועד! -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
