Bug #132187 for EAV-XS: t/08-file.t fails (and on some systems even more)

Sat Mar 21 05:24:42 2020 SREZIC [...] cpan.org - Ticket created

Subject:

t/08-file.t fails (and on some systems even more)

On some of my smokers t/08-file.t fails: ... # Failed test 'fail: bad@I♥NY.de' # at t/08-file.t line 33. # Failed test 'fail: bad@日本⒈co．jp' # at t/08-file.t line 33. # Looks like you failed 2 tests of 29. t/08-file.t ..... Dubious, test returned 2 (wstat 512, 0x200) Failed 2/29 subtests ... On others even more fails: ... # Failed test 'pass: 'иван@иванов.рф'' # at t/06-utf8.t line 45. # Failed test 'pass: 'user@ارامكو.ارامكو'' # at t/06-utf8.t line 45. # Failed test 'pass: 'r2l@عربي.de'' # at t/06-utf8.t line 45. # Failed test 'pass: '微博@微博.微博'' # at t/06-utf8.t line 45. # Failed test 'pass: 'الجزائر@الجزائر.الجزائر'' # at t/06-utf8.t line 45. # Failed test 'pass: 'ok@hello.vermögensberater'' # at t/06-utf8.t line 45. # Failed test 'pass: 'ok@クラウド.クラウド'' # at t/06-utf8.t line 45. # Failed test 'pass: 'ok@日本｡co｡jp'' # at t/06-utf8.t line 45. # Failed test 'pass: 'country-code@भारतम्.भारतम्'' # at t/06-utf8.t line 45. # Looks like you failed 9 tests of 24. t/06-utf8.t ..... Dubious, test returned 9 (wstat 2304, 0x900) Failed 9/24 subtests # Failed test 'pass: 'ok@дети.дети'' # at t/07-domains.t line 61. # Failed test 'pass: 'ok@ею.ею'' # at t/07-domains.t line 61. # Failed test 'pass: 'ok@қаз.қаз'' # at t/07-domains.t line 61. # Failed test 'pass: 'ok@קום.קום'' # at t/07-domains.t line 61. # Failed test 'pass: 'cc@ಭಾರತ.ಭಾರತ'' # at t/07-domains.t line 61. # Failed test 'pass: 'cc@ଭାରତ.ଭାରତ'' # at t/07-domains.t line 61. # Failed test 'pass: 'cc@ভাৰত.ভাৰত'' # at t/07-domains.t line 61. # Failed test 'pass: 'cc@भारोत.भारोत'' # at t/07-domains.t line 61. # Failed test 'pass: 'cc@भारतम्.भारतम्'' # at t/07-domains.t line 61. # Failed test 'pass: 'cc@بارت.بارت'' # at t/07-domains.t line 61. # Failed test 'pass: 'cc@ڀارت.ڀارت'' # at t/07-domains.t line 61. # Failed test 'pass: 'cc@ഭാരതം.ഭാരതം'' # at t/07-domains.t line 61. # Looks like you failed 12 tests of 40. t/07-domains.t .. Dubious, test returned 12 (wstat 3072, 0xc00) Failed 12/40 subtests # Failed test 'pass: проверка@тест.рф' # at t/08-file.t line 19. # Failed test 'pass: valid@faß.de' # at t/08-file.t line 19. # Failed test 'pass: valid@fäß.de' # at t/08-file.t line 19. # Failed test 'pass: valid@öbb.at' # at t/08-file.t line 19. # Failed test 'pass: valid@ÖBB.at' # at t/08-file.t line 19. # Failed test 'pass: valid@عربي.de' # at t/08-file.t line 19. # Failed test 'pass: valid@نامهای.de' # at t/08-file.t line 19. # Failed test 'pass: valid-mapping@日本｡co｡jp' # at t/08-file.t line 19. # Failed test 'pass: valid-mapping@日本｡co．jp' # at t/08-file.t line 19. # Failed test 'pass: lowercase@Σόλος.gr' # at t/08-file.t line 19. # Looks like you failed 10 tests of 29. t/08-file.t ..... Dubious, test returned 10 (wstat 2560, 0xa00) Failed 10/29 subtests ... I don't see any clear patterns regarding OS or perl version.

Sun Mar 22 04:17:10 2020 TVV [...] cpan.org - Correspondence added 5 min

Hello, Thanks for the report. Please, answer on questions below: On Sat Mar 21 05:24:42 2020, SREZIC wrote: Show quoted text

> On some of my smokers t/08-file.t fails: > > ... > # Failed test 'fail: bad@I♥NY.de' > # at t/08-file.t line 33. > > # Failed test 'fail: bad@日本⒈co．jp' > # at t/08-file.t line 33. > # Looks like you failed 2 tests of 29. > t/08-file.t ..... > Dubious, test returned 2 (wstat 512, 0x200) > Failed 2/29 subtests > ...

I assume (with 99% confidence) this is because these systems have libidn installed, but neither libidn2 nor idnkit. This is not a bug, because of the differences between IDNA 2003 (libidn) and IDNA 2008 (libidn2, idnkit). I know about the problem and this will be fixed in next releases. Show quoted text

> On others even more fails: > > ... > # Failed test 'pass: 'иван@иванов.рф'' > # at t/06-utf8.t line 45. > > # Failed test 'pass: 'user@ارامكو.ارامكو'' > # at t/06-utf8.t line 45. > > # Failed test 'pass: 'r2l@عربي.de'' > # at t/06-utf8.t line 45. > > # Failed test 'pass: '微博@微博.微博'' > # at t/06-utf8.t line 45. > > # Failed test 'pass: 'الجزائر@الجزائر.الجزائر'' > # at t/06-utf8.t line 45. > > # Failed test 'pass: 'ok@hello.vermögensberater'' > # at t/06-utf8.t line 45. > > # Failed test 'pass: 'ok@クラウド.クラウド'' > # at t/06-utf8.t line 45. > > # Failed test 'pass: 'ok@日本｡co｡jp'' > # at t/06-utf8.t line 45. > > # Failed test 'pass: 'country-code@भारतम्.भारतम्'' > # at t/06-utf8.t line 45. > # Looks like you failed 9 tests of 24. > t/06-utf8.t ..... > Dubious, test returned 9 (wstat 2304, 0x900) > Failed 9/24 subtests > > # Failed test 'pass: 'ok@дети.дети'' > # at t/07-domains.t line 61. > > # Failed test 'pass: 'ok@ею.ею'' > # at t/07-domains.t line 61. > > # Failed test 'pass: 'ok@қаз.қаз'' > # at t/07-domains.t line 61. > > # Failed test 'pass: 'ok@קום.קום'' > # at t/07-domains.t line 61. > > # Failed test 'pass: 'cc@ಭಾರತ.ಭಾರತ'' > # at t/07-domains.t line 61. > > # Failed test 'pass: 'cc@ଭାରତ.ଭାରତ'' > # at t/07-domains.t line 61. > > # Failed test 'pass: 'cc@ভাৰত.ভাৰত'' > # at t/07-domains.t line 61. > > # Failed test 'pass: 'cc@भारोत.भारोत'' > # at t/07-domains.t line 61. > > # Failed test 'pass: 'cc@भारतम्.भारतम्'' > # at t/07-domains.t line 61. > > # Failed test 'pass: 'cc@بارت.بارت'' > # at t/07-domains.t line 61. > > # Failed test 'pass: 'cc@ڀارت.ڀارت'' > # at t/07-domains.t line 61. > > # Failed test 'pass: 'cc@ഭാരതം.ഭാരതം'' > # at t/07-domains.t line 61. > # Looks like you failed 12 tests of 40. > t/07-domains.t .. > Dubious, test returned 12 (wstat 3072, 0xc00) > Failed 12/40 subtests > > # Failed test 'pass: проверка@тест.рф' > # at t/08-file.t line 19. > > # Failed test 'pass: valid@faß.de' > # at t/08-file.t line 19. > > # Failed test 'pass: valid@fäß.de' > # at t/08-file.t line 19. > > # Failed test 'pass: valid@öbb.at' > # at t/08-file.t line 19. > > # Failed test 'pass: valid@ÖBB.at' > # at t/08-file.t line 19. > > # Failed test 'pass: valid@عربي.de' > # at t/08-file.t line 19. > > # Failed test 'pass: valid@نامهای.de' > # at t/08-file.t line 19. > > # Failed test 'pass: valid-mapping@日本｡co｡jp' > # at t/08-file.t line 19. > > # Failed test 'pass: valid-mapping@日本｡co．jp' > # at t/08-file.t line 19. > > # Failed test 'pass: lowercase@Σόλος.gr' > # at t/08-file.t line 19. > # Looks like you failed 10 tests of 29. > t/08-file.t ..... > Dubious, test returned 10 (wstat 2560, 0xa00) > Failed 10/29 subtests > ... > > I don't see any clear patterns regarding OS or perl version.

Could you provide at least one full report for the cases above? And also tell me the IDN library information (name and version)? Thank you.

Sun Mar 22 04:17:11 2020 The RT System itself - Status changed from 'new' to 'open'

Mon Mar 23 02:26:10 2020 SREZIC [...] cpan.org - Correspondence added

On 2020-03-22 04:17:10, TVV wrote: Show quoted text

> Hello, > > Thanks for the report. Please, answer on questions below: > > On Sat Mar 21 05:24:42 2020, SREZIC wrote:

> > On some of my smokers t/08-file.t fails: > > > > ... > > # Failed test 'fail: bad@I♥NY.de' > > # at t/08-file.t line 33. > > > > # Failed test 'fail: bad@日本⒈co．jp' > > # at t/08-file.t line 33. > > # Looks like you failed 2 tests of 29. > > t/08-file.t ..... > > Dubious, test returned 2 (wstat 512, 0x200) > > Failed 2/29 subtests > > ...

> > > I assume (with 99% confidence) this is because these systems have > libidn installed, but neither libidn2 nor idnkit. This is not a bug, > because of the differences between IDNA 2003 (libidn) and IDNA 2008 > (libidn2, idnkit). I know about the problem and this will be fixed in > next releases.

I see for example my centos6 and centos8 smokers have these "short" fails. The centos7 smoker passes. However, centos8 has libidn2 installed: Name : libidn2 Version : 2.2.0 Release : 1.el8 Show quoted text

> >

> > On others even more fails: > > > > ... > > # Failed test 'pass: 'иван@иванов.рф'' > > # at t/06-utf8.t line 45. > > > > # Failed test 'pass: 'user@ارامكو.ارامكو'' > > # at t/06-utf8.t line 45. > > > > # Failed test 'pass: 'r2l@عربي.de'' > > # at t/06-utf8.t line 45. > > > > # Failed test 'pass: '微博@微博.微博'' > > # at t/06-utf8.t line 45. > > > > # Failed test 'pass: 'الجزائر@الجزائر.الجزائر'' > > # at t/06-utf8.t line 45. > > > > # Failed test 'pass: 'ok@hello.vermögensberater'' > > # at t/06-utf8.t line 45. > > > > # Failed test 'pass: 'ok@クラウド.クラウド'' > > # at t/06-utf8.t line 45. > > > > # Failed test 'pass: 'ok@日本｡co｡jp'' > > # at t/06-utf8.t line 45. > > > > # Failed test 'pass: 'country-code@भारतम्.भारतम्'' > > # at t/06-utf8.t line 45. > > # Looks like you failed 9 tests of 24. > > t/06-utf8.t ..... > > Dubious, test returned 9 (wstat 2304, 0x900) > > Failed 9/24 subtests > > > > # Failed test 'pass: 'ok@дети.дети'' > > # at t/07-domains.t line 61. > > > > # Failed test 'pass: 'ok@ею.ею'' > > # at t/07-domains.t line 61. > > > > # Failed test 'pass: 'ok@қаз.қаз'' > > # at t/07-domains.t line 61. > > > > # Failed test 'pass: 'ok@קום.קום'' > > # at t/07-domains.t line 61. > > > > # Failed test 'pass: 'cc@ಭಾರತ.ಭಾರತ'' > > # at t/07-domains.t line 61. > > > > # Failed test 'pass: 'cc@ଭାରତ.ଭାରତ'' > > # at t/07-domains.t line 61. > > > > # Failed test 'pass: 'cc@ভাৰত.ভাৰত'' > > # at t/07-domains.t line 61. > > > > # Failed test 'pass: 'cc@भारोत.भारोत'' > > # at t/07-domains.t line 61. > > > > # Failed test 'pass: 'cc@भारतम्.भारतम्'' > > # at t/07-domains.t line 61. > > > > # Failed test 'pass: 'cc@بارت.بارت'' > > # at t/07-domains.t line 61. > > > > # Failed test 'pass: 'cc@ڀارت.ڀارت'' > > # at t/07-domains.t line 61. > > > > # Failed test 'pass: 'cc@ഭാരതം.ഭാരതം'' > > # at t/07-domains.t line 61. > > # Looks like you failed 12 tests of 40. > > t/07-domains.t .. > > Dubious, test returned 12 (wstat 3072, 0xc00) > > Failed 12/40 subtests > > > > # Failed test 'pass: проверка@тест.рф' > > # at t/08-file.t line 19. > > > > # Failed test 'pass: valid@faß.de' > > # at t/08-file.t line 19. > > > > # Failed test 'pass: valid@fäß.de' > > # at t/08-file.t line 19. > > > > # Failed test 'pass: valid@öbb.at' > > # at t/08-file.t line 19. > > > > # Failed test 'pass: valid@ÖBB.at' > > # at t/08-file.t line 19. > > > > # Failed test 'pass: valid@عربي.de' > > # at t/08-file.t line 19. > > > > # Failed test 'pass: valid@نامهای.de' > > # at t/08-file.t line 19. > > > > # Failed test 'pass: valid-mapping@日本｡co｡jp' > > # at t/08-file.t line 19. > > > > # Failed test 'pass: valid-mapping@日本｡co．jp' > > # at t/08-file.t line 19. > > > > # Failed test 'pass: lowercase@Σόλος.gr' > > # at t/08-file.t line 19. > > # Looks like you failed 10 tests of 29. > > t/08-file.t ..... > > Dubious, test returned 10 (wstat 2560, 0xa00) > > Failed 10/29 subtests > > ... > > > > I don't see any clear patterns regarding OS or perl version.

> > Could you provide at least one full report for the cases above? And > also tell me the IDN library information (name and version)? >

FAILs: debian:10 (buster): libidn11/stable,now 1.33-2.2 amd64 [installed] Sample: http://www.cpantesters.org/cpan/report/43ebae36-6c70-11ea-a6a8-8a381f24ea8f freebsd12: libidn-1.35 Sample: http://www.cpantesters.org/cpan/report/339ad066-6c70-11ea-a6a8-8a381f24ea8f PASSes: debian:11 (bullseye): libidn11/testing,stable,now 1.33-2.2 amd64 [installed] libidn2-0/testing,now 2.3.0-1 amd64 [installed] fedora31: Name : libidn Version : 1.35 Release : 6.fc31 ... Name : libidn2 Version : 2.3.0 Release : 1.fc31 As usual, all reports may be seen at: http://matrix.cpantesters.org/?dist=EAV-XS%200.3.2

Mon Mar 23 02:33:39 2020 TVV [...] cpan.org - Correspondence added

On Mon Mar 23 09:26:10 2020, SREZIC wrote: Show quoted text

> As usual, all reports may be seen at: > http://matrix.cpantesters.org/?dist=EAV-XS%200.3.2

Hello. I saw them already and I have found what is problem too. This is because of these systems have the locale is not UTF-8 compatible. In that case, I assume, an IDN library is unable to correctly work. I have already made a patch, but if you have interest to test yourself, you may add this lines at top of t/06-utf8.t, t/07-domains.t, t/08-file.t files (after "use ..." directives): use POSIX qw(setlocale LC_ALL); setlocale(LC_ALL, "en_US.UTF-8"); Another way is call by hand "LC_ALL=en_US.UTF-8 make test". Either, before building and testing the module is set "export LC_ALL=en_US.UTF-8". Thanks.

Mon Mar 23 02:34:16 2020 SREZIC [...] cpan.org - Correspondence added

On 2020-03-23 02:26:10, SREZIC wrote: Show quoted text

> On 2020-03-22 04:17:10, TVV wrote:

> > Hello, > > > > Thanks for the report. Please, answer on questions below: > > > > On Sat Mar 21 05:24:42 2020, SREZIC wrote:

> > > On some of my smokers t/08-file.t fails: > > > > > > ... > > > # Failed test 'fail: bad@I♥NY.de' > > > # at t/08-file.t line 33. > > > > > > # Failed test 'fail: bad@日本⒈co．jp' > > > # at t/08-file.t line 33. > > > # Looks like you failed 2 tests of 29. > > > t/08-file.t ..... > > > Dubious, test returned 2 (wstat 512, 0x200) > > > Failed 2/29 subtests > > > ...

> > > > > > I assume (with 99% confidence) this is because these systems have > > libidn installed, but neither libidn2 nor idnkit. This is not a bug, > > because of the differences between IDNA 2003 (libidn) and IDNA 2008 > > (libidn2, idnkit). I know about the problem and this will be fixed in > > next releases.

> > I see for example my centos6 and centos8 smokers have these "short" > fails. The centos7 smoker passes. However, centos8 has libidn2 > installed: > > Name : libidn2 > Version : 2.2.0 > Release : 1.el8 >

> > > >

> > > On others even more fails: > > > > > > ... > > > # Failed test 'pass: 'иван@иванов.рф'' > > > # at t/06-utf8.t line 45. > > > > > > # Failed test 'pass: 'user@ارامكو.ارامكو'' > > > # at t/06-utf8.t line 45. > > > > > > # Failed test 'pass: 'r2l@عربي.de'' > > > # at t/06-utf8.t line 45. > > > > > > # Failed test 'pass: '微博@微博.微博'' > > > # at t/06-utf8.t line 45. > > > > > > # Failed test 'pass: 'الجزائر@الجزائر.الجزائر'' > > > # at t/06-utf8.t line 45. > > > > > > # Failed test 'pass: 'ok@hello.vermögensberater'' > > > # at t/06-utf8.t line 45. > > > > > > # Failed test 'pass: 'ok@クラウド.クラウド'' > > > # at t/06-utf8.t line 45. > > > > > > # Failed test 'pass: 'ok@日本｡co｡jp'' > > > # at t/06-utf8.t line 45. > > > > > > # Failed test 'pass: 'country-code@भारतम्.भारतम्'' > > > # at t/06-utf8.t line 45. > > > # Looks like you failed 9 tests of 24. > > > t/06-utf8.t ..... > > > Dubious, test returned 9 (wstat 2304, 0x900) > > > Failed 9/24 subtests > > > > > > # Failed test 'pass: 'ok@дети.дети'' > > > # at t/07-domains.t line 61. > > > > > > # Failed test 'pass: 'ok@ею.ею'' > > > # at t/07-domains.t line 61. > > > > > > # Failed test 'pass: 'ok@қаз.қаз'' > > > # at t/07-domains.t line 61. > > > > > > # Failed test 'pass: 'ok@קום.קום'' > > > # at t/07-domains.t line 61. > > > > > > # Failed test 'pass: 'cc@ಭಾರತ.ಭಾರತ'' > > > # at t/07-domains.t line 61. > > > > > > # Failed test 'pass: 'cc@ଭାରତ.ଭାରତ'' > > > # at t/07-domains.t line 61. > > > > > > # Failed test 'pass: 'cc@ভাৰত.ভাৰত'' > > > # at t/07-domains.t line 61. > > > > > > # Failed test 'pass: 'cc@भारोत.भारोत'' > > > # at t/07-domains.t line 61. > > > > > > # Failed test 'pass: 'cc@भारतम्.भारतम्'' > > > # at t/07-domains.t line 61. > > > > > > # Failed test 'pass: 'cc@بارت.بارت'' > > > # at t/07-domains.t line 61. > > > > > > # Failed test 'pass: 'cc@ڀارت.ڀارت'' > > > # at t/07-domains.t line 61. > > > > > > # Failed test 'pass: 'cc@ഭാരതം.ഭാരതം'' > > > # at t/07-domains.t line 61. > > > # Looks like you failed 12 tests of 40. > > > t/07-domains.t .. > > > Dubious, test returned 12 (wstat 3072, 0xc00) > > > Failed 12/40 subtests > > > > > > # Failed test 'pass: проверка@тест.рф' > > > # at t/08-file.t line 19. > > > > > > # Failed test 'pass: valid@faß.de' > > > # at t/08-file.t line 19. > > > > > > # Failed test 'pass: valid@fäß.de' > > > # at t/08-file.t line 19. > > > > > > # Failed test 'pass: valid@öbb.at' > > > # at t/08-file.t line 19. > > > > > > # Failed test 'pass: valid@ÖBB.at' > > > # at t/08-file.t line 19. > > > > > > # Failed test 'pass: valid@عربي.de' > > > # at t/08-file.t line 19. > > > > > > # Failed test 'pass: valid@نامهای.de' > > > # at t/08-file.t line 19. > > > > > > # Failed test 'pass: valid-mapping@日本｡co｡jp' > > > # at t/08-file.t line 19. > > > > > > # Failed test 'pass: valid-mapping@日本｡co．jp' > > > # at t/08-file.t line 19. > > > > > > # Failed test 'pass: lowercase@Σόλος.gr' > > > # at t/08-file.t line 19. > > > # Looks like you failed 10 tests of 29. > > > t/08-file.t ..... > > > Dubious, test returned 10 (wstat 2560, 0xa00) > > > Failed 10/29 subtests > > > ... > > > > > > I don't see any clear patterns regarding OS or perl version.

> > > > Could you provide at least one full report for the cases above? And > > also tell me the IDN library information (name and version)? > >

> > FAILs: > > debian:10 (buster): > libidn11/stable,now 1.33-2.2 amd64 [installed] > Sample: http://www.cpantesters.org/cpan/report/43ebae36-6c70-11ea- > a6a8-8a381f24ea8f > > freebsd12: > libidn-1.35 > Sample: http://www.cpantesters.org/cpan/report/339ad066-6c70-11ea- > a6a8-8a381f24ea8f > > PASSes: > > debian:11 (bullseye): > libidn11/testing,stable,now 1.33-2.2 amd64 [installed] > libidn2-0/testing,now 2.3.0-1 amd64 [installed] > > fedora31: > Name : libidn > Version : 1.35 > Release : 6.fc31 > ... > Name : libidn2 > Version : 2.3.0 > Release : 1.fc31 > > > As usual, all reports may be seen at: > http://matrix.cpantesters.org/?dist=EAV-XS%200.3.2

And here's the summary per OS/distribution (only smoke reports from my machines): PASS: - ubuntu 18.04 (bionic) - ubuntu 19.10 (eoan) - debian 11 (bullseye) - centos 7 - fedora 31 FAIL: - ubuntu 16.04 (xenial) - debian 7 (wheezy) - debian 9 (stretch) - debian 10 (buster) - centos 6 - centos 8 - freebsd 10 - freebsd 12 - freebsd 13 UNKNOWN: - freebsd 9

Mon Mar 23 03:03:31 2020 SREZIC [...] cpan.org - Correspondence added

On 2020-03-23 02:33:39, TVV wrote: Show quoted text

> On Mon Mar 23 09:26:10 2020, SREZIC wrote:

> > As usual, all reports may be seen at: > > http://matrix.cpantesters.org/?dist=EAV-XS%200.3.2

> > Hello. I saw them already and I have found what is problem too. This > is because of these systems have the locale is not UTF-8 compatible. > In that case, I assume, an IDN library is unable to correctly work. > > I have already made a patch, but if you have interest to test > yourself, you may add this lines at top of t/06-utf8.t, t/07- > domains.t, t/08-file.t files (after "use ..." directives): > > use POSIX qw(setlocale LC_ALL); > setlocale(LC_ALL, "en_US.UTF-8"); > > Another way is call by hand "LC_ALL=en_US.UTF-8 make test". Either, > before building and testing the module is set "export > LC_ALL=en_US.UTF-8". >

Indeed: env LC_ALL=en_US.UTF-8 cpan_smoke_modules -skiptestedpass EAV-XS improved the situation. With the changed locale it passes on FreeBSD (10, 12, 13) and Debian 10.

Mon Mar 23 21:27:48 2020 TVV [...] cpan.org - Correspondence added

Fixed in 0.3.3.

Mon Mar 23 21:27:49 2020 TVV [...] cpan.org - Status changed from 'open' to 'resolved'

Mon Mar 23 21:27:49 2020 TVV [...] cpan.org - Taken

Mon Mar 23 21:27:49 2020 TVV [...] cpan.org - Fixed in 0.3.3 added