Bug #121072 for clearpress: Clearpress application failures under Apache httpd mpm-worker

Tue Apr 11 09:45:34 2017 mg8 [...] sanger.ac.uk - Ticket created

Subject:	Clearpress application failures under Apache httpd mpm-worker
Date:	Tue, 11 Apr 2017 14:18:36 +0100
To:	bug-clearpress [...] rt.cpan.org
From:	Marina Gourtovaia <mg8 [...] sanger.ac.uk>

Under mpm-worker multiple threads are using the same Perl interpreter. Please see https://github.com/wtsi-npg/npg_tracking/pull/389 for a fix, which configures mpm-worker to be more like mpm-prefork OS Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-75-generic x86_64) Apache httpd version 2.4.23 mod_perl version 2.0.10 Perl version 5.22.2 compiled to use threads -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

Fri Apr 14 05:16:27 2017 rmp [...] psyphi.net - Correspondence added

Hi Marina, Very interesting, Thanks. I knew it didn't work under mpm_worker (only mpm_prefork) but I'd ever pinned it to Config::IniFiles - will look into a fix. R. On Tue Apr 11 09:45:34 2017, mg8@sanger.ac.uk wrote: Show quoted text

> Under mpm-worker multiple threads are using the same Perl interpreter. > Please see https://github.com/wtsi-npg/npg_tracking/pull/389 for a fix, > which configures mpm-worker to be more like mpm-prefork > > OS Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-75-generic x86_64) > Apache httpd version 2.4.23 > mod_perl version 2.0.10 > Perl version 5.22.2 compiled to use threads > >

Fri Apr 14 05:16:27 2017 The RT System itself - Status changed from 'new' to 'open'

Fri Apr 14 06:46:08 2017 mg8 [...] sanger.ac.uk - Correspondence added

Subject:	Re: [rt.cpan.org #121072] Clearpress application failures under Apache httpd mpm-worker
Date:	Fri, 14 Apr 2017 11:45:50 +0100
To:	bug-clearpress [...] rt.cpan.org
From:	Marina Gourtovaia <mg8 [...] sanger.ac.uk>

Hi Roger Hope you are well On 14/04/2017 10:16, Roger Pettett via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=121072 > > > Hi Marina, > > Very interesting, Thanks. I knew it didn't work under mpm_worker (only mpm_prefork) but I'd ever pinned it to Config::IniFiles - will look into a fix.

Thanks for looking into this. Initially I though to rewrite the util (I could have brought all Clearpress util into our util and changed it), but then decided that there is no guarantee that all other CPAN dependencies Clearpress or our code are using will behave. So we configured mpm-worker to use up to 50 children, one thread per child effectively reducing it to what mpm-prefork does. Since it's impossible to switch between prefork and worker without recompilation and we wanted to maintain one httpd executable between all our applications and worker is better for some of them, we decided to stick to worker and use this strange configuration of ours. Regards Marina Show quoted text

> > R. > > On Tue Apr 11 09:45:34 2017, mg8@sanger.ac.uk wrote:

>> Under mpm-worker multiple threads are using the same Perl interpreter. >> Please see https://github.com/wtsi-npg/npg_tracking/pull/389 for a fix, >> which configures mpm-worker to be more like mpm-prefork >> >> OS Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-75-generic x86_64) >> Apache httpd version 2.4.23 >> mod_perl version 2.0.10 >> Perl version 5.22.2 compiled to use threads >> >>

> >

-- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

Mon Jun 26 12:20:49 2017 rmp [...] psyphi.net - Correspondence added

I don't believe this is the fault of Config::IniFiles. I think it's do to with the $INSTANCES management in ClearPress::util which has classically assumed on request per-process. I just pushed a change to #dev branch which uses the Apache request object to assign a per-request UUID and key off that when available. "Classic" mpm_prefork behaviour should be preserved. Note that if you haven't used recent versions, other recent changes modify response-header handling considerably but that behaviour should be unrelated to this. I haven't done extensive testing with this dev version yet but I believe the same change should be functional under both mpm_worker and mpm_event. R. On Tue Apr 11 09:45:34 2017, mg8@sanger.ac.uk wrote: Show quoted text

> Under mpm-worker multiple threads are using the same Perl interpreter. > Please see https://github.com/wtsi-npg/npg_tracking/pull/389 for a fix, > which configures mpm-worker to be more like mpm-prefork > > OS Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-75-generic x86_64) > Apache httpd version 2.4.23 > mod_perl version 2.0.10 > Perl version 5.22.2 compiled to use threads > >

Mon Jun 26 12:36:56 2017 rmp [...] psyphi.net - Taken

Mon Jun 26 12:56:02 2017 mg8 [...] sanger.ac.uk - Correspondence added

Subject:	Re: [rt.cpan.org #121072] Clearpress application failures under Apache httpd mpm-worker
Date:	Mon, 26 Jun 2017 17:37:30 +0100
To:	bug-clearpress [...] rt.cpan.org
From:	Marina Gourtovaia <mg8 [...] sanger.ac.uk>

Hi Roger On 26/06/2017 17:20, Roger Pettett via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=121072 > > > I don't believe this is the fault of Config::IniFiles.

Recently, I've seen all sorts of random failures. Show quoted text

> I think it's do to with the $INSTANCES management in ClearPress::util which has classically assumed on request per-process. > > I just pushed a change to #dev branch which uses the Apache request object to assign a per-request UUID and key off that when available. "Classic" mpm_prefork behaviour should be preserved.

Thanks for this. I'll retest everything mid-July. In particular, it would be interesting to see if the latest httpd server will work with new changes; we had to stop on 2.4.23 Marina -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

Tue Jul 18 16:16:36 2017 rmp [...] psyphi.net - Correspondence added

appears stable in: - 2.4.26 macOS - 2.4.18 linux (ubuntu xenial) - 2.4.7 linux (ubuntu trusty)

Tue Jul 18 16:16:36 2017 rmp [...] psyphi.net - Status changed from 'open' to 'resolved'

Tue Jul 18 16:16:36 2017 rmp [...] psyphi.net - Fixed in v476.1.1 added