On Sat, Feb 16, 2013 at 06:55:09PM +1100, Dave Chinner wrote: > On Fri, Feb 15, 2013 at 08:59:06PM +0000, Hugo Mills wrote: > > Hi, Hemanth, > > > > Here's a question -- what are you testing? (Not just here, but in > > general, with your test infrastructure) > > > > There are (at least) three classes of tests you could be doing: > > > > 1) Unit tests, which test individual functions within the code and > > ensure they do what they're meant to do. > > > > 2) Integration tests, which test the full end-to-end system. > > > > 3) Partial integration tests, which exercise the kernel > > filesystem code. > > > > 4) Partial integration tests, which exercise the userspace tools code. > > > > > > Now, clearly you're not doing (1) here. It's going to be hard to > > separate (2) from (3) and (4), but it's possible to write your tests > > to do more of one or of the other. (*) > > > > xfstests clearly is much more geared to (3), and stresses the > > kernel filesystem implementation rather than the userspace tools. If > > Definitely not. There are lots of userspace tools tests in xfstests > for stuff like mkfs.xfs, xfs_repair, xfs_dump/restore, > xfs_quota, xfs_metadump/restore, xfs_copy, etc. OK, this is my mistake. I guess I've basically seen so many kernel-side things fail from xfstests, I'd assumed that was exclusively what it was testing. Thanks for the correction. > > you want to test the implementation of qgroups, it belongs in > > xfstests. If you want to test the userspace code, you need to make > > sure that (over all your tests) you cover every command-line option, > > and every different way of using the tool, and ensure that it does the > > right things. > > Yup, that's why there are are so many xfs specific tests. :) e.g. > there are 31 tests that use xfsdump/restore in lots of different > ways, including exercising dumping to tape devices if you have the > hardware... [snip] > Seriously, guys, just write new tests for xfstests. Everything you > need to run btrfs-progs tests is already there. Don't try to > re-invent the wheel simply because you don't understand how the > current wheels we have are made.... As long as xfstests already has tests for userspace tools in it, then I'm quite happy to see it all go there. My point about being clear exactly which bit of the end-to-end system the test is there to exercise still stands, though. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- He's playing Schubert. I think Schubert is losing. ---
Attachment:
signature.asc
Description: Digital signature
