Goffredo Baroncelli posted on Sat, 03 Oct 2015 19:41:33 +0200 as excerpted: > On 2015-10-03 12:09, Axel Burri wrote: >> >> >> On 2015-10-03 11:56, Goffredo Baroncelli wrote: >>> On 2015-10-02 18:41, axel@xxxxxxx wrote: >>>> Old implementation used tabs "\t", and tried to work around problems >>>> by guessing amount of tabs needed (e.g. "\t\t" after top level", with >>>> buggy output as soon as empty uuids are printed). This will never >>>> work correctly, as tab width is a user-defined setting in the >>>> terminal. >>> >>> >>> Why not use string_table() and table_*() functions ? >> >> string_table(), as well as all table functions by nature, needs to know >> the maximum size of all cells in a row before printing, and therefore >> buffers all the output before printing. It would eat up a lot of memory >> for large tables (it is not unusual to have 1000+ subvolumes in btrfs >> if you make heavy use of snapshotting). Furthermore, it would slow down >> things by not printing the output linewise. > > > Assuming 200bytes per row (== subvolume) x 1000 subvolumes = 200kB... I > don't think that this could be a problem, nor in terms of memory used > nor in terms of speed: if you have 1000+ subvolumes the most time > consuming activity is traversing the filesystem looking at the > snapshot... Perhaps unfortunately, scaling to millions of snapshots/subvolumes really *is* expected by some people. You'd be surprised at the number of folks that setup automated per-minute snapshotting with no automated thinning, and expect to be able to keep several years' worth of snapshots, and then wonder why btrfs maintenance commands such as balance take weeks/months... 5 years * 365 days/year * 24 hours/day * 60 minutes/hour * 1 snapshot/ minute = ... (years, days, hours, minutes cancel out, leaving snapshots... yeah, the math should be right) 2.6 million plus snapshots. And that's just one subvolume. Multiply that by a half dozen subvolumes... Over 15 million snapshots. Multiply that by 200 bytes/snapshot... Several gigs of buffer memory, now. Obviously btrfs doesn't scale to that level now, and if it did, someone making the mistake of trying to get a listing of millions of snapshots would very likely change their mind before even hitting 10%... But that's why actually processing line-by-line is important, so they'll actually /see/ what happened and ctrl-C it, instead of the program aborting as it runs into (for example) the 32-bit user/kernel memory barrier, without printing anything useful... The line-by-line way... "Oops, that's waayyy too many snapshots to be practical, maybe I should start thinning..." The buffer-it-all-way... "Oops, there's a bug in the program; it crashed." -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
