Bug #42531 for RT-Authen-ExternalAuth: Authentication failure due to system error doesn't really fail in 0.07

Mon Jan 19 10:24:26 2009 cpanbugs-20070819 [...] billmail.scconsult.com - Ticket created

Subject:

Authentication failure due to system error doesn't really fail in 0.07_02

Using rt 3.8.2 and ExternalAuth 0.07_02 on FreeBSD 7.0 in a dedicated jail, dependencies installed from ports but RT installed by hand and ExternalAuth from CPAN. No code or template customizations so far other than ExternalAuth. The web server is: Apache/2.2.11 (FreeBSD) mod_ssl/2.2.11 OpenSSL/0.9.8e DAV/2 mod_perl/2.0.4 Perl/v5.8.8 Using LDAP for authentication, a correct username with a record in LDAP or internally and correct password works as expected on the first test after an Apache restart: unknown users have an unprivileged account created, unprivileged accounts are sent to SelfService, privileged accounts go to the home page. However, subsequent logins display a disturbing behavior. Rather than succeeding or failing or falling back to RT internal auth, a "System error" page is sent back (see attachment "goodpasserr.txt") with an error apparently inside the Net::LDAP complex and the following in the RT log (logging at 'debug' depth via syslog) Jan 17 22:39:28 rttest RT: $pass defined (MUNGE), Running IsPassword Jan 17 22:39:28 rttest RT: Trying External Authentication ( root ) Jan 17 22:39:28 rttest RT: Attempting to use external auth service: My_LDAP Jan 17 22:39:28 rttest RT: Trying external auth service: My_LDAP Jan 17 22:39:28 rttest RT: LDAP Search === Base: dc=com == Filter: (&(mail=root)(objectClass=inetOrgPerson)) == Attrs: dn As you see, the failure is before there's even success or failure from the search for the user DN, so there should be no chance of success in authentication (and in that case the user is internal and could not succeed in LDAP.) Yet after that error, reloading the base URL results in a redirect to the "At A Glance" home page, because there's a new RT_SID cookie that arrived *on the error page* that somehow RT is trusting as identifying an authenticated session. Obviously this is a complex bug. It has taken me some time to nail down the behavior, as it appears to be a combination of problems in RT proper, ExternalAuth, and perl-ldap, with mod_perl a likely contributor to the perl-ldap failure that exposed the problem. I may end up reporting bugs in many places on this, but I figured the top layer was the best place to start.

Subject:

goodpasserr.txt

Message body is not shown because it is too large.

Wed Jan 21 06:39:40 2009 zordrak [...] cpan.org - Correspondence added

Can you please advise the version of perl-ldap in use? Also, can you re-test against v0.08_01?

Wed Jan 21 06:39:40 2009 The RT System itself - Status changed from 'new' to 'open'

Wed Jan 21 12:07:51 2009 cpanbugs-20070819 [...] billmail.scconsult.com - Correspondence added

On Wed Jan 21 06:39:40 2009, ZORDRAK wrote: Show quoted text

> Can you please advise the version of perl-ldap in use?

perl-ldap 0.39 I also have another illuminating line from the Apache error log, specifying the "import failure" at the root of the crash inside the LDAP Search call: [Sat Jan 17 22:39:28 2009] [warning]: "LDAP_SYNC_INFO" is not exported by the Net::LDAP::Constant module at /usr/local/lib/perl5/site_perl/5.8.8/Net/LDAP/Intermediate.pm line 10 (/usr/local/lib/perl5/5.8.8/Carp.pm:271) That's not a problem I can reproduce outside of mod_perl2, it is nonsensical based on the Constant.pm code, and as I mentioned this does not occur on the first authentication attempt after an Apache restart. Switching to FastCGI eliminated the crash, and I can run a trivial LDAP search script that goes down the same call path, and running it with 'perl -d' I can step right through the code that is failing under mod_perl2. It seems clear to me that the cause of the crash itself is mod_perl2, not any of the stack of modules being run by it. However, the top-level problem is not the crash, but rather the fact that a crash deep down a calling chain topped by ExternalAuth trying to search for a user DN results in the user getting back a session cookie that RT treats as if there had been an actual authentication. Show quoted text

> Also, can you re-test against v0.08_01?

Probably. I did install it and get it working (with the workaround mentioned in bug 42584) and I will probably have time to flip back to mod_perl2 today for a test. Of course, the weird nature of the mod_perl-dependent error smells to me like the sort of thing that is likely to be sensitive to obscure infrastructural trivia, so I would not be shocked if I can't get the same exact error. It would still be important to understand how the generic case of a fatal error inside ExternalAuth can result in what RT deems to be an authenticated session.

Wed Jan 21 12:48:03 2009 zordrak [...] cpan.org - Correspondence added

On Wed Jan 21 12:07:51 2009, cpanbugs-20070819@billmail.scconsult.com wrote: Show quoted text

> On Wed Jan 21 06:39:40 2009, ZORDRAK wrote:

> > Can you please advise the version of perl-ldap in use?

> > perl-ldap 0.39 > > I also have another illuminating line from the Apache error log, > specifying the "import > failure" at the root of the crash inside the LDAP Search call: > > [Sat Jan 17 22:39:28 2009] [warning]: "LDAP_SYNC_INFO" is not exported > by the > Net::LDAP::Constant module at > /usr/local/lib/perl5/site_perl/5.8.8/Net/LDAP/Intermediate.pm line 10 > (/usr/local/lib/perl5/5.8.8/Carp.pm:271)

This is *definitely* worth posting on the perl-ldap mailing list as far as I can see. I've had problems with it before when Graham released a version that overloaded the result codes with policy errors so that any time you checked a Constant (such as 0) you wouldn't get a "SUCCESS" or other normal constant you'd get a "PP_POLICY_xxx" result instead. Definitely sounds like something's been overlooked. Show quoted text

> However, the top-level problem is not the crash, but rather the fact > that a crash deep down > a calling chain topped by ExternalAuth trying to search for a user DN > results in the user > getting back a session cookie that RT treats as if there had been an > actual authentication.

I understand. To be honest, I'm not sure quite what's going on there. Certainly as far as I know all the calls out to Net::LDAP are error-checked, but not to the point of eval :/

Thu Jan 22 12:55:25 2009 cpanbugs-20070819 [...] billmail.scconsult.com - Correspondence added

On Wed Jan 21 12:48:03 2009, ZORDRAK wrote: Show quoted text

> On Wed Jan 21 12:07:51 2009, cpanbugs-20070819@billmail.scconsult.com wrote:

> > On Wed Jan 21 06:39:40 2009, ZORDRAK wrote:

> > > Can you please advise the version of perl-ldap in use?

> > > > perl-ldap 0.39 > > > > I also have another illuminating line from the Apache error log, > > specifying the "import > > failure" at the root of the crash inside the LDAP Search call: > > > > [Sat Jan 17 22:39:28 2009] [warning]: "LDAP_SYNC_INFO" is not exported > > by the > > Net::LDAP::Constant module at > > /usr/local/lib/perl5/site_perl/5.8.8/Net/LDAP/Intermediate.pm line 10 > > (/usr/local/lib/perl5/5.8.8/Carp.pm:271)

> > This is *definitely* worth posting on the perl-ldap mailing list as far > as I can see.

Indeed. I'll probably get around to that in my copious free time. Show quoted text

> > However, the top-level problem is not the crash, but rather the fact > > that a crash deep down > > a calling chain topped by ExternalAuth trying to search for a user DN > > results in the user > > getting back a session cookie that RT treats as if there had been an > > actual authentication.

> > I understand. To be honest, I'm not sure quite what's going on there. > Certainly as far as I know all the calls out to Net::LDAP are > error-checked, but not to the point of eval :/

Well, whatever is different in 0.08_01 fixed that problem as far as ExternalAuth is concerned: running under mod_perl I still get a psychotic failure in the perl-ldap world, and the error page carries a RT_SID cookie, but a reload does not get past authentication. FWIW, now the error is usually the same one cited in my first message, but if I pound repeatedly on the reload I can also get through sometimes (logs show auth actually occurring correctly) and sometimes I get this stack dump instead: System error error: Can't locate object method "new" via package "Net::LDAP::Search" at /usr/local/lib/perl5/site_perl/5.8.8/Net/LDAP.pm line 257. context: ... 253: } 254: 255: sub message { 256: my $ldap = shift; 257: shift->new($ldap, @_); 258: } 259: 260: sub async { 261: my $ldap = shift; ... code stack: /usr/local/lib/perl5/site_perl/5.8.8/Net/LDAP.pm:257 /usr/local/lib/perl5/site_perl/5.8.8/Net/LDAP.pm:438 /opt/rt3/local/plugins/RT-Authen-ExternalAuth/lib/RT/Authen/ExternalAuth/LDAP.pm:312 /opt/rt3/local/plugins/RT-Authen-ExternalAuth/lib/RT/Authen/ExternalAuth.pm:356 /opt/rt3/local/plugins/RT-Authen-ExternalAuth/lib/RT/Authen/ExternalAuth.pm:106 /opt/rt3/local/plugins/RT-Authen-ExternalAuth/html/Callbacks/ExternalAuth/autohandler/Auth:25 /opt/rt3/share/html/autohandler:256 I have not actually looked in Net/LDAP/Search.pm, but I'm willing to bet that the constructor isn't randomly disappearing and reappearing depending on how fast the app is getting hit.... Definitely looks like a bug in mod_perl.

Thu Jan 22 12:55:26 2009 cpanbugs-20070819 [...] billmail.scconsult.com - Status changed from 'open' to 'resolved'

Bug #42531 for RT-Authen-ExternalAuth: Authentication failure due to system error doesn't really fail in 0.07_02