On Mon Aug 06 17:47:29 2018, JKEENAN wrote:
Show quoted text> On Mon Aug 06 13:55:43 2018, JKEENAN wrote:
> > As Test::Smoke run, it logs output in 'smokecurrent.log'.
> >
> > #####
> > $ ls -l smokecurrent.log
> > -rw-r--r-- 1 jkeenan jkeenan 603104 Aug 4 21:56 smokecurrent.log
> > #####
> >
> > At the conclusion of a run, it creates 4 files in the
> > logs/smokecurrent subdirectory, the names of which files are composed
> > from the SHA of the commit being tested.
> >
> > #####
> > $ ls -ltr logs/smokecurrent/ | tail -4
> > -rw-rw-rw- 1 jkeenan jkeenan 1454 Aug 4 21:56
> > rpte94c3f6a8d515b39961010524037ca044f029524.rpt
> > -rw-rw-rw- 1 jkeenan jkeenan 2371 Aug 4 21:56
> > oute94c3f6a8d515b39961010524037ca044f029524.out
> > -rw-rw-rw- 1 jkeenan jkeenan 595939 Aug 4 21:56
> > loge94c3f6a8d515b39961010524037ca044f029524.log
> > -rw-rw-rw- 1 jkeenan jkeenan 6471 Aug 4 21:56
> > jsne94c3f6a8d515b39961010524037ca044f029524.jsn
> > #####
> >
> > "Naively" -- that is, as a typical user who has not peered deeply
> > into
> > the Test::Smoke code base or documentation -- I would expect that
> > ./smokecurrent.log and ./logs/smokecurrent/log***.log to be either
> > (a)
> > exactly the same file or (b) to be almost exactly the same but differ
> > in some clear, well thought-out way. By observation over repeated
> > Test::Smoke runs, I infer that (b) is usally the case and that some
> > data is deliberately and thoughtfully removed from ./smokecurrent.log
> > before ./logs/smokecurrent/log***.log is written.
> >
> > What I am finding instead is that the data in
> > ./logs/smokecurrent/log***.log is simply incomplete and defective.
> > It
> > appears that the data from the top log file breaks off in mid-line as
> > its being written to the lower log file!
> >
> > I am attaching 4 gzipped files which demonstrate the problem: 2 from
> > a Test::Smoke run on Linux; 2 from a run on FreeBSD. The problem
> > appears on both platforms.
> >
> > #####
> > $ ls -l
> > total 16
> > -rw-r--r-- 1 jkeenan jkeenan 2540 Aug 6 13:37
> > freebsd.smokecurrent.log.tail.gz
> > -rw-r--r-- 1 jkeenan jkeenan 1837 Aug 6 13:37
> > freebsd.logs.smokecurrent.loge94c3f.log.tail.gz
> > -rw-r--r-- 1 jkeenan jkeenan 2323 Aug 6 13:33
> > linode.smokecurrent.log.tail.gz
> > -rw-r--r-- 1 jkeenan jkeenan 1671 Aug 6 13:33
> > linode.logs.smokecurrent.loge94c3f.log.tail.gz
> > #####
> >
> > For each of the 4 files provided I took the original file and deleted
> > everything before the line containing "Finished smoking". On each
> > platform the file whose name ends in "smokecurrent.log.tail.gz" is an
> > extract from ./smokecurrent.log (the "top-level" log file); the file
> > whose name ends in logs.smokecurrent.loge94c3f.log.tail.gz is an
> > extract from
> > ./logs/smokecurrent/loge94c3f6a8d515b39961010524037ca044f029524.log
> > (the "lower-level" log file).
> >
> > In each platform-specific pair, you will observe that:
> >
> > * The top-level log file is complete, i.e., it displays all of the
> > .json data, the URL of the report posting at perl5.test-smoke.org,
> > and
> > the archiver messages.
> >
> > * The lower-level report breaks off mid-line in the middle of the
> > .json data. The URL of the report posting at perl5.test-smoke.org is
> > never reached, nor are the archiver messages.
> >
> > This is a serious defect. Each test run overwrites
> > ./smokecurrent.log, so the only data available for archiving for
> > future study is that found in ./logs/smokecurrent/log***.log. If
> > that
> > data is incomplete, the human smoke-tester is not equipped to
> > properly
> > answer questions about particular Test::Smoke runs.
> >
> > Thank you very much.
> > Jim Keenan
>
> On further inspection, I notice that a typical "top-level"
> ./smokecurrent.log file ends with text like this:
>
> #####
> [2018-08-06 16:18:36-0400]
http://perl5.test-smoke.org/report/68640
> [2018-08-06 16:18:36-0400] ==> Starting archiver
> [2018-08-06 16:18:36-0400] Copy(/home/jkeenan/p5smoke/perl-
> current/mktest.rpt,
> /home/jkeenan/p5smoke/logs/smokecurrent/rptd342516410da8af9d1824e50179d7d6ff3c7ba95.rpt):
> ok
> [2018-08-06 16:18:36-0400] Copy(/home/jkeenan/p5smoke/perl-
> current/mktest.out,
> /home/jkeenan/p5smoke/logs/smokecurrent/outd342516410da8af9d1824e50179d7d6ff3c7ba95.out):
> ok
> [2018-08-06 16:18:36-0400] Copy(/home/jkeenan/p5smoke/perl-
> current/mktest.jsn,
> /home/jkeenan/p5smoke/logs/smokecurrent/jsnd342516410da8af9d1824e50179d7d6ff3c7ba95.jsn):
> ok
> [2018-08-06 16:18:36-0400]
> Copy(/home/jkeenan/p5smoke/smokecurrent.log,
> /home/jkeenan/p5smoke/logs/smokecurrent/logd342516410da8af9d1824e50179d7d6ff3c7ba95.log):
> ok
> #####
>
> The last line is a message wherein we log the successful copy of
> smokecurrent.log to logs/smokecurrent/log***.log. But that's
> premature, as we're still writing to the file we want to copy. What
> is probably happening is that the a certain portion of the file is
> being copied at the point that the command to copy is issued, but the
> additional log messages created thereafter are not part of the copied
> text.
>
> archive_log() (lib/Test/Smoke/Archiver.pm line 194) is called by
> archive_files() (same file line 85), which in turn is called by run()
> (lib/Test/Smoke/App/Archiver.pm line 44), which is probably called by
> run() (lib/Test/Smoke/App/SmokePerl.pm line 114), which in turn is
> probably called by bin/tssmokeperl.pl.
>
> Perhaps archive_log() should not be called as part of
> Test::Smoke::Archiver::archive_files(). We should log a message like
> "Preparing to copy log file", close the handle writing to
> ./smokecurrent.log, copy the file and then *append* a line to
> ./logs/smokecurrent/log***.log indicating that we have successfully
> copied the log file. Granted, that would make a 1-line difference
> between the top-level and lower-level log files -- but that would be a
> minor flaw compared to what we currently have.
>
Today I have been reviewing Test-Smoke-1.72_06.
#####
[install] $ ack '^\$VERSION' ../Test-Smoke-1.72_06/lib/Test/Smoke.pm
$VERSION = '1.72_06';
#####
I believe that it has not corrected the problem of a truncated logs/smokecurrent/log*.log file reported here in RT 126010.
I conducted 4 smoke test runs using 1.72_06. If you go to
http://perl5.test-smoke.org/search and enter 'freebsd' for OS and 'perlmonger.nycbug.org' for host, the 4 runs executed on 2018-11-14 are the runs in question.
#####
Arch Host OS Git-id Branch Smoke date Status URL
amd64 perlmonger.nycbug.org freebsd - 11.2-STABLE v5.28.0-47-g94027dc85b maint-5.28 2018-11-14 21:47:08+00 PASS
http://perl5.test-smoke.org/report/74039
amd64 perlmonger.nycbug.org freebsd - 11.2-STABLE v5.26.2-85-gadf1b5d28a maint-5.26 2018-11-14 21:11:06+00 PASS
http://perl5.test-smoke.org/report/74035
amd64 perlmonger.nycbug.org freebsd - 11.2-STABLE v5.29.4-42-g61edc94a36 blead 2018-11-14 17:33:51+00 FAIL(F)
http://perl5.test-smoke.org/report/74026
amd64 perlmonger.nycbug.org freebsd - 11.2-STABLE v5.29.1-31-gd3bb03e084 undefined 2018-11-14 15:26:08+00 FAIL(F)
http://perl5.test-smoke.org/report/74020
#####
(Also in .csv attachment.)
(The "Status" grades are not relevant, as there were test failures in blead over the course of the day.)
(The "undefined" value for Branch in v5.29.1-31-gd3bb03e084 is not very relevant, as that was a commit I checked out to test
https://rt.cpan.org/Ticket/Display.html?id=125932.)
I am attaching the tails of the 'smokecurrent.log' files for each of these 4 smoke-test runs. For convenience, I'm also attaching a file which cats those 4 tails into a single file, as well as the perl -V output for the last of the 4 smoke-test runs (maint-5.28). Note that in each of these tails there is no line logging the "Copy" of smokecurrent.log as "logs/smokecurrent/log*.log". So the archived version of the log is therefore incomplete.
The pre-archived smokecurrent.log file does, however, have a "Copy" line in it. Example:
#####
[install] $ diff smokecurrent.log logs/smokecurrent/log94027dc85b896cfea4bd017b273f22e472882d68.log |less
1483d1482
< [2018-11-14 22:16:16+0000] Copy(/usr/home/jkeenan/learn/perl/test-smoke/rtc-125932/install/smokecurrent.log, /usr/home/jkeenan/learn/perl/test-smoke/rtc-125932/install/logs/smokecurrent/log94027dc85b896cfea4bd017b273f22e472882d68.log): ok
#####
Thank you very much.
Jim Keenan