On 2019/9/12 下午10:12, Nikolay Borisov wrote:
>
>
> On 12.09.19 г. 17:03 ч., David Newall wrote:
>> Hello Qu,
>>
>> Thank you very much for helping me with this.
>>
>> On 12/9/19 4:35 pm, Qu Wenruo wrote:
>>> Would you please check how fast (or how slow in this particular case)
>>> the related disks are?
>>> To me, it really looks like just too slow devices.
>>
>> I discover that you are correct about the underlying storage being
>> slow. Nikolay suggested that, too.
>>
>> Although I mentioned that the filesystem is encrypted with luks on the
>> VM, I didn't say that the underlying storage is connected via multipath
>> iSCSI (with two paths) on the host server, and provided to the VM via
>> KVM as Virtio disk, which should be fine, but, using dd (bs=1024k
>> count=15) on the VM, I'm seeing a woeful 255KB/s read speed through the
>> encryption layer, and 274KB/s from the raw disk. :-(
>>
>> On the host, I'm seeing 2MB/s via one path and 846KB/s via the other, so
>> I think that's where I need to turn my attention. (Time to benchmark,
>> turn off one path, and speak to the DC management.)
>>
>>> I see all dumps are waiting for write_all_supers.
>>>
>>> Would you please provide the code context of
>>> write_all_supers.isra.43+0x977?
>>>
>>> I guess it's wait_dev_flush(), which is just really waiting for disk
>>> writes.
>>
>> Sorry, I don't understand what you mean by "code context". Maybe the
>> question is now moot.
>>
>> Although it's now apparent that I've got a really slow disk, I still
>> wonder if btrfs is holding a lock for an unnecessarily long time
>> (assuming that it is btrfs holding the lock.) I feel that having to
>> wait tens of minutes to find the device names of mounted devices could
>> never be intended, so there might be something that needs tweaking.
>>
>
> With the kernel you are using that's how things were structured for
> various reasons. Recent kernel versions do not take device_list_mutex
> when printing the device name. So if you update your kernel to one which
> contains :
>
> 88c14590cdd6 ("btrfs: use RCU in btrfs_show_devname for device list
> traversal")
Oh, we have already addressed that in a better way than rw_semaphore...
I must have lived under a rock for a while...
Thanks,
Qu
>
> this particular problem would be gone. Looking at the history of that
> commit this means kernel 4.17 at least.
>
> <snip>
>
>>
