Bug #100337 for App-cpanminus-reporter: a failing result for CPAN.pm was submitted as a PASS

Explicitly referencing this log file resulted in the same thing, so it's definitely parsing this log and not a different one: $; cpanm-reporter --force --build_logfile=/Users/ether/.cpanm/work/1415990050.50326/build.log sending: (http://mirrors.gossamer-threads.com/CPAN/authors/id/A/AN/ANDK/CPAN-2.05.tar.gz, ANDK, CPAN-2.05, PASS)

Hi! So, this is a real bug in cpanm-reporter, but it's sort of an unsolvable one - at least until we can get it really merged into cpanm. See, the failing report includes a "Result: PASS" line in the middle of the test, and from reading the build.log we can't really tell it apart from an actual "Result: PASS". I have discussed this at length with renormalist and we came to the conclusion that no matter what you try and parse, there could always be a module that would print the string we're parsing and we would not be able to tell it apart from the actual cpanm testing output :( The solution could be any of the following: 1) Merge cpanm-reporter to cpanm so we always fetch the test output and will never have to parse anything anymore. This is being discussed on https://github.com/miyagawa/cpanminus/pull/285. It is the final goal for cpanm-reporter, but it might need some work to get it going the way we need. 2) Blacklist on the parser all modules that are known to produce bad reports. 3) Special-case on the parser all modules that known to produce bad reports. #1 is of course our main goal, but like I said it might need some work :/ What's our second best option?