btrfs-progs-5.6-1.fc32.x86_64
kernel-5.6.8-300.fc32.x86_64
I've created a new file system, filled it with ~600G, set the seed
flag, and I'm creating a sprout by adding a 2nd device and removing
the 1st. This used to go at device speeds. Today it's running at
~35M/s where I expect 95-120M/s. I don't know that it's related to
send/receive performance.
The remove process with 'perf top -g -U' is only using 7% CPU, unlike
the send/receive cases.
Samples
Children Self Shared Objec Symbol
- 82.39% 0.04% [kernel] [k] relocate_block_group
- 17.88% relocate_block_group
- 7.78% prepare_to_relocate
- 7.21% btrfs_commit_transaction
- 7.51% btrfs_run_delayed_refs
- __btrfs_run_delayed_refs
- 3.19% __tracepoint_kvm_mmu_spte_requested
- 3.15% pipapo_get.constprop.0
- 3.57% __add_to_free_space_tree
3.61% btrfs_del_items
1.37% btrfs_insert_empty_items
- 1.41% __tracepoint_kvm_entry
0.92% __tracepoint_kvm_apic_accept_irq
- 6.41% relocate_data_extent
- 7.52% relocate_file_extent_cluster
- 2.35% __do_page_cache_readahead
- 1.70% read_pages.isra.0
- 2.03% extent_readpages
1.32% __do_readpage
0.93% __alloc_pages_nodemask
- 1.67% clear_extent_bit
1.92% __clear_extent_bit
- 1.40% lock_extent_bits
1.59% __set_extent_bit
1.19% btrfs_delalloc_reserve_metadata
1.15% __set_page_dirty_nobuffers
- 2.23% prepare_to_merge
- 1.84% btrfs_commit_transaction
- 2.15% btrfs_run_delayed_refs
- __btrfs_run_delayed_refs
1.47% __btrfs_inc_extent_ref.isra.0
+ 38.75% 0.47% [kernel] [k] __btrfs_run_delayed_refs
+ 32.98% 0.21% [kernel] [k] relocate_file_extent_cluster
+ 29.88% 0.00% [kernel] [k] relocate_data_extent
+ 21.63% 0.00% libc-2.31.so [.] __GI___ioctl
+ 21.46% 0.00% [kernel] [k] entry_SYSCALL_64_after_hwframe
+ 21.46% 0.00% [kernel] [k] do_syscall_64
+ 21.46% 0.00% [kernel] [k] __x64_sys_ioctl
+ 21.46% 0.00% [kernel] [k] ksys_ioctl
+ 21.46% 0.00% [kernel] [k] rpc_info_open
+ 21.46% 0.00% [kernel] [k] btrfs_rm_device
For a higher level overview, try: perf top --sort comm,dso
And also running perf doesn't improve performance, which it does
rather remarkably for send/receive.
--
Chris Murphy
