Bug #110276 for Test-Expect: Nondeterministic test failures

As seen on the CPAN testers list, t/simple.t sometimes fails test 10 with # Failed test 'expect' # at lib/Test/Expect.pm line 86. # got: 'ping # pong' # expected: 'pong' # Looks like you failed 1 test of 18. t/simple.t .. I can reproduce this locally by loading the host with other I/O and the running the test in a loop. It looks like it's normally reading out the 'pong' response from the previous 'expect_send' test (rather than its own) that never gets drained. However, sometimes stderr from careless argument handling in the 'read' script intervenes: read: Can't open world: No such file or directory at ./read line 11. I've got strace output for the failures and can provide it if you like, but I think just applying the attached two proposed patches should fix it. Thanks for your work on Test-Expect, -- Niko Tyni (Debian Perl Group) ntyni@debian.org

From c49a4c13fa543618dab322a57952fa60cbe6e67e Mon Sep 17 00:00:00 2001 From: Niko Tyni <ntyni@debian.org> Date: Thu, 10 Dec 2015 22:29:57 +0200 Subject: [PATCH 1/2] Fix testscripts not to treat their argument as a file The argument is intended to be the name of the peer, not a file to be read. --- read | 2 +- readline | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/read b/read index 56252cc..6eb6a34 100644 --- a/read +++ b/read @@ -3,7 +3,7 @@ use strict; use warnings; $| = 1; -my $what = $ARGV[0] || "there"; +my $what = (shift @ARGV) || "there"; print "* Hi $what, to read\n"; while (1) { diff --git a/readline b/readline index df5665c..4445981 100644 --- a/readline +++ b/readline @@ -3,7 +3,7 @@ use strict; use warnings; use Term::ReadLine; -my $what = $ARGV[0] || "there"; +my $what = (shift @ARGV) || "there"; print "* Hi $what, to readline\n"; my $term = Term::ReadLine->new('readline'); -- 2.6.2

From 477835d918f294acb26a53eeeeb0ab7ab4cb3a53 Mon Sep 17 00:00:00 2001 From: Niko Tyni <ntyni@debian.org> Date: Thu, 10 Dec 2015 22:32:06 +0200 Subject: [PATCH 2/2] Read out the buffer after expect_send This and the previous commit fix nondeterministic test failures where output from the previous expect_send and stderr from the argument handling can make the response vary. --- t/simple.t | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/t/simple.t b/t/simple.t index 841dda4..4217207 100644 --- a/t/simple.t +++ b/t/simple.t @@ -3,7 +3,7 @@ use strict; use warnings; use lib 'lib'; use Test::Expect; -use Test::More tests => 18; +use Test::More tests => 20; require_ok('Expect'); @@ -20,5 +20,6 @@ foreach my $filename ('read', 'readline') { expect_like(qr/Hi world, to $filename/, "expect_like"); expect_is("* Hi world, to $filename", "expect_is"); expect_send("ping", "expect_send"); + expect_is("pong", "expect_is again"); expect("ping", "pong", "expect"); }; -- 2.6.2