Hi there buttery folks, I just hit this warning and oops running a parallel fs_mark create workload on a test VM using a 17TB btrfs filesystem (12 disk dm RAID0) using default mkfs and mount parmeters, mounted on /mnt/scratch. The VM has 8p and 4GB RAM, and the fs_mark command line was: $ ./fs_mark -D 10000 -S0 -n 100000 -s 0 -L 250 \ -d /mnt/scratch/0 -d /mnt/scratch/1 \ -d /mnt/scratch/2 -d /mnt/scratch/3 \ -d /mnt/scratch/4 -d /mnt/scratch/5 \ -d /mnt/scratch/6 -d /mnt/scratch/7 The attached image should give you a better idea of the performance drop-off that was well under way when the crash occurred at about 96 million files created. I'm rerunning the test on a fresh filesystem, so I guess I'll see if this a one-off in the next couple of hours.... Cheers, Dave. [70972.788725] device fsid 5c8b0884-4e94-41ac-9a7e-8628a03a6f6d devid 1 transid 4 /dev/vdc [70972.795441] btrfs: disk space caching is enabled [121727.817810] ------------[ cut here ]------------ [121727.819430] WARNING: at fs/btrfs/extent-tree.c:4771 __btrfs_free_extent+0x630/0x6d0() [121727.821729] Hardware name: Bochs [121727.823417] Modules linked in: [121727.824601] Pid: 2984, comm: btrfs-transacti Not tainted 3.2.0-rc7-dgc+ #167 [121727.826784] Call Trace: [121727.827508] [<ffffffff8108a69f>] warn_slowpath_common+0x7f/0xc0 [121727.829290] [<ffffffff8108a6fa>] warn_slowpath_null+0x1a/0x20 [121727.830785] [<ffffffff815b0680>] __btrfs_free_extent+0x630/0x6d0 [121727.832486] [<ffffffff815ac110>] ? block_group_cache_tree_search+0x90/0xc0 [121727.833907] [<ffffffff815b42f1>] run_clustered_refs+0x381/0x800 [121727.834627] [<ffffffff815b483a>] btrfs_run_delayed_refs+0xca/0x220 [121727.836118] [<ffffffff815c5469>] btrfs_commit_transaction+0x359/0x840 [121727.837042] [<ffffffff810ac420>] ? add_wait_queue+0x60/0x60 [121727.838053] [<ffffffff815c5da4>] ? start_transaction+0x94/0x2b0 [121727.839133] [<ffffffff810986d0>] ? usleep_range+0x50/0x50 [121727.840642] [<ffffffff815be0d5>] transaction_kthread+0x265/0x280 [121727.842918] [<ffffffff815bde70>] ? btrfs_congested_fn+0xb0/0xb0 [121727.844480] [<ffffffff810ab97c>] kthread+0x8c/0xa0 [121727.845772] [<ffffffff81abb5f4>] kernel_thread_helper+0x4/0x10 [121727.847204] [<ffffffff810ab8f0>] ? flush_kthread_worker+0xa0/0xa0 [121727.848803] [<ffffffff81abb5f0>] ? gs_change+0x13/0x13 [121727.850073] ---[ end trace 804317a2ff1cf200 ]--- [121727.851207] btrfs unable to find ref byte nr 14368411648 parent 0 root 2 owner 0 offset 0 [121727.853324] BUG: unable to handle kernel NULL pointer dereference at (null) [121727.855261] IP: [<ffffffff815e60f2>] map_private_extent_buffer+0x12/0x150 [121727.856730] PGD 0 [121727.857105] Oops: 0000 [#1] SMP [121727.857243] CPU 1 [121727.857243] Modules linked in: [121727.857243] [121727.857243] Pid: 2984, comm: btrfs-transacti Tainted: G W 3.2.0-rc7-dgc+ #167 Bochs Bochs [121727.857243] RIP: 0010:[<ffffffff815e60f2>] [<ffffffff815e60f2>] map_private_extent_buffer+0x12/0x150 [121727.857243] RSP: 0018:ffff880117b49b00 EFLAGS: 00010286 [121727.857243] RAX: 0000000000000000 RBX: 0000000000000065 RCX: ffff880117b49b30 [121727.857243] RDX: 0000000000000004 RSI: 000000000000007a RDI: 0000000000000000 [121727.857243] RBP: ffff880117b49b20 R08: ffff880117b49b38 R09: ffff880117b49b40 [121727.857243] R10: 0000000000000000 R11: 0000000000000001 R12: 000000000000007a [121727.857243] R13: 0000000000000000 R14: 00000000ffffffe4 R15: 0000000000001000 [121727.857243] FS: 0000000000000000(0000) GS:ffff88011fc40000(0000) knlGS:0000000000000000 [121727.857243] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [121727.857243] CR2: 0000000000000000 CR3: 0000000001ec1000 CR4: 00000000000006e0 [121727.857243] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [121727.857243] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [121727.857243] Process btrfs-transacti (pid: 2984, threadinfo ffff880117b48000, task ffff88010f38e280) [121727.857243] Stack: [121727.857243] 00000000ffffffe4 0000000000000065 000000000000007a 0000000000000000 [121727.857243] ffff880117b49b70 ffffffff815db96c ffff880117b49b80 ffff880117b49b40 [121727.857243] 0000000000000002 00000003586c9000 0000000000000000 ffff8801042d97f0 [121727.857243] Call Trace: [121727.857243] [<ffffffff815db96c>] btrfs_item_size+0x3c/0x90 [121727.857243] [<ffffffff815b0282>] __btrfs_free_extent+0x232/0x6d0 [121727.857243] [<ffffffff815ac110>] ? block_group_cache_tree_search+0x90/0xc0 [121727.857243] [<ffffffff815b42f1>] run_clustered_refs+0x381/0x800 [121727.857243] [<ffffffff815b483a>] btrfs_run_delayed_refs+0xca/0x220 [121727.857243] [<ffffffff815c5469>] btrfs_commit_transaction+0x359/0x840 [121727.857243] [<ffffffff810ac420>] ? add_wait_queue+0x60/0x60 [121727.857243] [<ffffffff815c5da4>] ? start_transaction+0x94/0x2b0 [121727.857243] [<ffffffff810986d0>] ? usleep_range+0x50/0x50 [121727.857243] [<ffffffff815be0d5>] transaction_kthread+0x265/0x280 [121727.857243] [<ffffffff815bde70>] ? btrfs_congested_fn+0xb0/0xb0 [121727.857243] [<ffffffff810ab97c>] kthread+0x8c/0xa0 [121727.857243] [<ffffffff81abb5f4>] kernel_thread_helper+0x4/0x10 [121727.857243] [<ffffffff810ab8f0>] ? flush_kthread_worker+0xa0/0xa0 [121727.857243] [<ffffffff81abb5f0>] ? gs_change+0x13/0x13 [121727.857243] Code: 83 c0 01 48 89 85 78 ff ff ff e9 c0 fc ff ff 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 55 41 54 53 48 83 ec 08 66 66 66 66 90 <4c> 8b 27 48 89 [121727.857243] RIP [<ffffffff815e60f2>] map_private_extent_buffer+0x12/0x150 [121727.857243] RSP <ffff880117b49b00> [121727.857243] CR2: 0000000000000000 [121727.904461] ---[ end trace 804317a2ff1cf201 ]--- -- Dave Chinner david@xxxxxxxxxxxxx
Attachment:
xfs-btrfs-ext4.png
Description: PNG image
