Bug #32826 for LWP-Authen-Negotiate: Problem connecting NTLM-protected corporate site

Thu Jan 31 05:28:22 2008 kushpet [...] mail.ru - Ticket created

Subject:

Problem connecting NTLM-protected corporate site

Hi! I am using: Perl, v5.10.0 built for MSWin32-x86-multi-thread GSSAPI-0.24 LWP-Authen-Negotiate-0.06 Authen-NTLM-0.31 kfw-3-2-2.exe Microsoft Windows XP Professional Version 2002 Service Pack 2 in corporate network and try to access NTLM-protected site from LWP. The first response from site has HTTP header like this: Hypertext Transfer Protocol HTTP/1.1 401 Unauthorized\r\n Request Version: HTTP/1.1 Response Code: 401 Content-Length: 1656 Content-Type: text/html\r\n Server: Microsoft-IIS/6.0\r\n WWW-Authenticate: Negotiate\r\n WWW-Authenticate: NTLM\r\n MicrosoftOfficeWebServer: 5.0_Pub\r\n X-Powered-By: ASP.NET\r\n Date: Mon, 14 Jan 2008 10:36:56 GMT\r\n \r\n My program, when it goes down to call to $status = $ctx->init( $creds, $target, $imech, $iflags, $itime , $bindings,$itoken, undef, $otoken, undef, undef); in Negotiate.pm, prints the following: LWP::Authen::Negotiate::authenticate: Unspecified GSS failure. Minor code may provide more information LWP::Authen::Negotiate::authenticate: Server not found in Kerberos database and stops - does not proceed any further to NTLM::authenticate. I guess that this is because a server I would like to reach does not use Kerberos, but NTLM instead. So it is not listed in domain Kerberos database. (I can just guess, I do not have access to domain database). So why Negotiate doesn't process this scenario properly - skip to NTLM and form NTLM response? Thank you! BR, Dmitry

Thu Jan 31 14:57:47 2008 achim [...] grolmsnet.de - Correspondence added

Subject:	Re: [rt.cpan.org #32826] Problem connecting NTLM-protected corporate site
Date:	Thu, 31 Jan 2008 20:56:48 +0100
To:	bug-LWP-Authen-Negotiate [...] rt.cpan.org
From:	Achim Grolms <achim [...] grolmsnet.de>

On Thursday 31 January 2008, Dmitry Kushpet via RT wrote: Show quoted text

> So why Negotiate doesn't process this scenario properly - skip to NTLM > and form NTLM response?

Because LWP::Authen::Negotiate implements the Authentication by Kerberos only. Use LWP::Authen::NTLM instead if you want to authenticate by NTLM. Does this solve your problem? BR, Achim

Thu Jan 31 14:57:49 2008 The RT System itself - Status changed from 'new' to 'open'

Fri Feb 01 04:54:18 2008 kushpet [...] mail.ru - Correspondence added

Subject:	Re[2]: [rt.cpan.org #32826] Problem connecting NTLM-protected corporate site
Date:	Fri, 01 Feb 2008 12:54:00 +0300
To:	bug-LWP-Authen-Negotiate [...] rt.cpan.org
From:	Dmitriy Kushpet <kushpet [...] mail.ru>

Achim, thank you for your response! I am not too much experienced in Perl, Microsoft or cryptography. Just some considerations. May be, this is a problem of LWP::UserAgent? The problem is that I may not change server's behaviour. And it sends me an authentication header with 2 lines simultaneousely: WWW-Authenticate: Negotiate\r\n WWW-Authenticate: NTLM\r\n This header is then processed by LWP::UserAgent::request in a loop: unless ($class->can("authenticate")) { $response->header("Client-Warning" => "Unsupported authentication scheme '$scheme'"); next CHALLENGE; } return $class->authenticate($self, $proxy, $challenge, $response, $request, $arg, $size); So, to inhibit "Negotiate", I need to edit LWP::UserAgent::request like this: unless (($class->can("authenticate")) and (not ($scheme =~ /negotiate/))) { $response->header("Client-Warning" => "Unsupported authentication scheme '$scheme'"); next CHALLENGE; } return $class->authenticate($self, $proxy, $challenge, $response, $request, $arg, $size); That looks strange for me - just to ignore something that server is asking for. Anyway, the result is negative - see attached WireShark logs. I am afraid, that LWP::UserAgent needs to process properly two authentication schemes together. Otherwise, no way to use Perl in my case. BR, Dmitry Show quoted text

-----Original Message----- From: "achim@grolmsnet.de via RT" <bug-LWP-Authen-Negotiate@rt.cpan.org> To: kushpet@mail.ru Date: Thu, 31 Jan 2008 14:57:50 -0500 Subject: Re: [rt.cpan.org #32826] Problem connecting NTLM-protected corporate site

> > > <URL: http://rt.cpan.org/Ticket/Display.html?id=32826 > > > On Thursday 31 January 2008, Dmitry Kushpet via RT wrote: >

> > So why Negotiate doesn't process this scenario properly - skip to NTLM > > and form NTLM response?

> > Because LWP::Authen::Negotiate implements the Authentication > by Kerberos only. > > Use LWP::Authen::NTLM instead if you want to authenticate > by NTLM. > > Does this solve your problem? > > BR, > Achim >

Download from Perl no negotiate with NTLM fil.pcap
application/octet-stream 16.9k

Message body not shown because it is not plain text.

Download from_InternetExplorer fil.pcap
application/octet-stream 7.1k

Message body not shown because it is not plain text.

Fri Feb 01 06:56:32 2008 leifj [...] it.su.se - Correspondence added

Subject:	Re: [rt.cpan.org #32826] Problem connecting NTLM-protected corporate site
Date:	Fri, 01 Feb 2008 12:55:36 +0100
To:	bug-LWP-Authen-Negotiate [...] rt.cpan.org
From:	Leif Johansson <leifj [...] it.su.se>

Dmitry Kushpet via RT wrote: Show quoted text

> Queue: LWP-Authen-Negotiate > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=32826 > > > Achim, > > thank you for your response! > > I am not too much experienced in Perl, Microsoft or cryptography. Just some considerations. May be, this is a problem of LWP::UserAgent? > > The problem is that I may not change server's behaviour. And it sends me an authentication header with 2 lines simultaneousely: > > WWW-Authenticate: Negotiate\r\n > WWW-Authenticate: NTLM\r\n >

That is perfectly normal behaviour. That is the way HTTP does authentication mechanism negotiation. LWP::UserAgent imho needs a way for the user to specify policy about which mechanism to prefer given multiple offered ones. Cheers Leif

Fri Feb 01 14:29:42 2008 achim [...] grolmsnet.de - Correspondence added

Subject:	Re: [rt.cpan.org #32826] Problem connecting NTLM-protected corporate site
Date:	Fri, 1 Feb 2008 20:29:14 +0100
To:	bug-LWP-Authen-Negotiate [...] rt.cpan.org
From:	Achim Grolms <achim [...] grolmsnet.de>

On Friday 01 February 2008, Dmitry Kushpet via RT wrote: Show quoted text

> Queue: LWP-Authen-Negotiate > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=32826 > > > Achim, > > thank you for your response! > > I am not too much experienced in Perl, Microsoft or cryptography. Just some > considerations. May be, this is a problem of LWP::UserAgent?

What happens if you de-install LWP::Authen::Negotiate? In that case LWP::Authen::NTLM would work fine, I think. Does this help? In general: LWP::Authen::Negotiate does not deal with any cryptography, it is a very thein Layer passing the GSSAPI-tokens to a underlying GSSAPI-Implementation (Like MIT Kerberos or Heimdal). That means LWP::Authen::Negotiate supports the GSSAPI-mechnisms the underlying GSSAPI-implemantation supports. Does this help? BR, Achim

Fri Feb 01 14:34:23 2008 leifj [...] it.su.se - Correspondence added

Subject:	Re: [rt.cpan.org #32826] Problem connecting NTLM-protected corporate site
Date:	Fri, 01 Feb 2008 20:34:01 +0100
To:	bug-LWP-Authen-Negotiate [...] rt.cpan.org
From:	Leif Johansson <leifj [...] it.su.se>

achim@grolmsnet.de via RT wrote: Show quoted text

> Queue: LWP-Authen-Negotiate > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=32826 > > > On Friday 01 February 2008, Dmitry Kushpet via RT wrote: >

>> Queue: LWP-Authen-Negotiate >> Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=32826 > >> >> Achim, >> >> thank you for your response! >> >> I am not too much experienced in Perl, Microsoft or cryptography. Just some >> considerations. May be, this is a problem of LWP::UserAgent? >>

> > What happens if you > > de-install LWP::Authen::Negotiate? > > In that case LWP::Authen::NTLM would work fine, > I think. > > Does this help? > > In general: > > LWP::Authen::Negotiate does not deal with any cryptography, > it is a very thein Layer passing the GSSAPI-tokens > to a underlying GSSAPI-Implementation (Like MIT Kerberos or Heimdal). > > That means LWP::Authen::Negotiate supports the GSSAPI-mechnisms > the underlying GSSAPI-implemantation supports. > > Does this help? >

I don't think any of this helps actually - LWP needs to be able to let the user choose between mechanisms. This isn't an issue of LWP::Authen::* though. Cheers Leif

Fri Feb 01 14:40:58 2008 achim [...] grolmsnet.de - Correspondence added

Subject:	Re: [rt.cpan.org #32826] Problem connecting NTLM-protected corporate site
Date:	Fri, 1 Feb 2008 20:40:35 +0100
To:	bug-LWP-Authen-Negotiate [...] rt.cpan.org
From:	Achim Grolms <achim [...] grolmsnet.de>

On Friday 01 February 2008, Leif Johansson via RT wrote: Show quoted text

> I don't think any of this helps actually - LWP needs to be able to let > the user > choose between mechanisms.

But one way to choose.. is what LWP::Authen modules are installed. But you think of something like a configuration (per targethost/domain/whatever) or an API to set preferred authentication-Method? BR, Achim

Fri Feb 01 18:26:19 2008 leifj [...] it.su.se - Correspondence added

Subject:	Re: [rt.cpan.org #32826] Problem connecting NTLM-protected corporate site
Date:	Fri, 01 Feb 2008 22:35:45 +0100
To:	bug-LWP-Authen-Negotiate [...] rt.cpan.org
From:	Leif Johansson <leifj [...] it.su.se>

achim@grolmsnet.de via RT wrote: Show quoted text

> Queue: LWP-Authen-Negotiate > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=32826 > > > On Friday 01 February 2008, Leif Johansson via RT wrote: > >

>> I don't think any of this helps actually - LWP needs to be able to let >> the user >> choose between mechanisms. >>

> > But one way to choose.. is what LWP::Authen modules > are installed. > > But you think of something like a configuration > (per targethost/domain/whatever) > or an API to set preferred authentication-Method? > > BR, > Achim > >

I think it is unreasonable to ask the user to de-install NTLM or Negotiate as a way to select which to choose in a given situation. Compare with firefox where you list "trusted" domains/url:s for NTLM and Negotiate separately - and this is a *user* setting! I suspect you need to provide a callback to LWP::UserAgent that is given a list of authentication choices and returns the preferred one (you'll give it the URL etc too as a basis for the decision probably) in the case when there are multiple choices. In the absence of a provided callback there should be a default ordering between the common mechanisms which reflects some idea of relative strength, Cheers Leif