Re: [PATCH 1/5] sparse: Show expected vs. actual output on test failure
|[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]|
On Fri, 26 Aug 2011, Christopher Li wrote:
Your patch now makes the runner verbose for "known to fail" tests which is definitely not something we should do. If someone tagged the test as "known to fail", we should treat it just like we treat passed test cases.That is intentional. I haven't seen you reply on my comment so I take the liberty to change it. I do want to see the "known to fail" test case.
Sorry about that. I somehow missed the email.
The whole point of my patch was to make "make check" pinpoint *unexpected* breakage so that anyone who bothers to do "make check" on their patches can never cause regressions.How to you know your new patch did not break "known to fail" test case in a different way than originally? It could be regression as well.
I think that's irrelevant, really. I happen to think it's completely pointless to even run test cases that are "known to fail" because it creates confusion and adds little value.
And if we really care that a "known to fail" test case fails in a certain way, we can turn it into a "known _not_ to fail" and assert the exact failure case.
You can diff the aggregate output of "make check" before and after the new patch to see if the new patch affect any test case at all. That is better than blindly skip the "known to fail" test case.
No, that doesn't scale at all if you run "make check" every few minutes like I do.
Yes, I have a different usage case in mind. I want to look at what currently breaks. In the long run, hopefully we fix all the known issues so this wouldn't be any issue at all.
You can submit a patch to add a config for it if you are so insist on the "I don't care about already broken test cases" usage case. To me, diff the "make check" output is straightly better.
OK, a flag would definitely work for me. Pekka -- To unsubscribe from this list: send the line "unsubscribe linux-sparse" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
[Newbies FAQ] [Kernel List] [Site Home] [IETF Annouce] [DCCP] [Netdev] [Networking] [Security] [Bugtraq] [Photo] [Yosemite] [MIPS Linux] [ARM Linux] [Linux Security] [Linux RAID] [Linux SCSI] [DDR & Rambus] [Trinity Fuzzer Tool]