On Mon, Aug 19, 2019 at 08:47:28AM -0400, Josef Bacik wrote:
> > > +static inline u64 btrfs_calc_insert_metadata_size(struct btrfs_fs_info *fs_info,
> > > + unsigned num_items)
> > > {
> > > return (u64)fs_info->nodesize * BTRFS_MAX_LEVEL * 2 * num_items;
> > > }
> >
> > Isn't assuming that we are going to split on every level of the cow
> > rather pessimistic, bordering on impossible. Isn't it realistically
> > possible that we will only ever split up until root->level.
This is a natural idea, I had it as well long time ago, and your
argument why this does not work is that the level can increase between
the time of reservatino and time of commit. So this would lead to ENOSPC
where not expected and then abort.
> I had this discussion with Chris when I messed with this. We do pass in the
> root we intend on messing with so we could very well do this, but it sort of
> scares me because maybe we've been using more of our worst case reservations
> than I expected.
>
> My plan is to get these last corners worked out, get a few more months of
> production testing, and then start experimenting with using the root->level + 1
> for the maximum level and see how that goes. The ramp up from 1 level to 3
> level roots happens pretty fast, so I suspect there'll be weird corner cases
> going from empty->not empty, but I _think_ we should be fine to make this change
> further down the road?
I had a patch for that and tried to do worst case calculations for each
level, ie. what's the number minimum number of items that would require
splitting and increasing the level.
Starting with level 4, the +1 should be safe, for 5+ safe unless eg. the
commit period is so long that the number of dirty metadata is out of
scale (the period affected by commit=, so not a typical usecase).
So the whole idea should work. The formula I used was
level = max(4, root->level + 1)
instead of BTRFS_MAX_LEVEL.
For start, setting the minimum to 5 would be IMHO safe enough, the
savings in the over-reservations would be 3*nodesize in most cases, or
~38%. This could help to avoid early ENOSPC.
