Stefan Priebe - Profihost AG posted on Mon, 15 Jan 2018 10:55:42 +0100 as excerpted: > since around two or three years i'm using btrfs for incremental VM > backups. > > some data: > - volume size 60TB > - around 2000 subvolumes > - each differential backup stacks on top of a subvolume > - compress-force=zstd > - space_cache=v2 > - no quote / qgroup > > this works fine since Kernel 4.14 except that i need ssd_spread as an > option. If i do not use ssd_spread i always end up with very slow > performance and a single kworker process using 100% CPU after some days. > > With ssd_spread those boxes run fine since around 6 month. Is this > something expected? I haven't found any hint regarding such an impact. My understanding of the technical details is "limited" as I'm not a dev, and I expect you'll get a more technically accurate response later, but sometimes a first not particularly technical response can be helpful as long as it's not /wrong/. (And if it is this is a good way to have my understanding corrected as well. =:^) With that caveat, based on my understanding of what I've seen on-list... The kernel v4.14 ssd mount-option changes apparently primarily affected data, not metadata. Apparently, ssd_spread has a heavier metadata effect, and the v4.14 changes moved additional (I believe metadata) functionality to ssd-spread that had originally been part of ssd as well. There has been some discussion of metadata tweaks similar to those in 4.14 for the ssd option with data, but they weren't deemed as demonstrably needed as the ssd option tweaks and needed further discussion, so were put off until the effect of the 4.14 tweaks could be gauged in more widespread use, after which they were to be reconsidered, if necessary. Meanwhile, in the discussion I saw, Chris Mason mentioned that Facebook is using ssd-spread for various reasons there, so it's well-tested with their deployments, which I'd assume have many of the same qualities yours do, thus implying that your observations about ssd-spread are no accident. In fact, if I interpreted Chris's comments correctly, they use ssd_spread on very large multi-layered non-ssd storage arrays, in part because the larger layout-alignment optimizations make sense there as well as on ssds. That would appear to be precisely what you are seeing. =:^) If that's the case, then arguably the option is misnamed and the ssd_spread name may well at some point be deprecated in favor of something more descriptive of its actual function and target devices. Purely my own speculation here, but perhaps something like vla_spread (very-large- array)? -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
