Bug #69411 for HTTP-Proxy: libcurl-based downloads are failing

Mon Jul 11 18:15:33 2011 JTOPJIAN [...] cpan.org - Ticket created

Subject:

libcurl-based downloads are failing

Hello, I have a small HTTP::Proxy script that I use for caching yum and rpm repositories. With the release of RHEL6/CentOS6, I have found that proxy requests are failing. How to reproduce: Server side: A bare minimum HTTP::Proxy: $proxy = HTTP::Proxy->new( port => 8080, host => undef, engine => 'Threaded', ); $proxy->start; Client side: export http_proxy=http://192.168.1.1:8080 # curl -O http://mirror.utexas.edu/epel/6/x86_64/repodata/repomd.xml % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 101 4363 101 4363 0 0 14179 0 --:--:-- --:--:-- --:--:-- 14211 curl: (18) transfer closed with outstanding read data remaining # curl --http1.0 -O http://mirror.utexas.edu/epel/6/x86_64/repodata/repomd.xml % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 101 4363 101 4363 0 0 14826 0 --:--:-- --:--:-- --:--:-- 14890 Note that with the --http1.0 option, the download succeeds. wget-based downloads work fine, but since yum is using libcurl and pycurl as of RHEL6, yum no longer works with HTTP::Proxy. Thanks, Joe

Tue Apr 21 16:44:59 2015 book [...] cpan.org - Correspondence added

On Mon Jul 11 18:15:33 2011, JTOPJIAN wrote: Show quoted text

> Hello, > > I have a small HTTP::Proxy script that I use for caching yum and rpm > repositories. With the release of RHEL6/CentOS6, I have found that proxy > requests are failing. > > How to reproduce: > > Server side: > A bare minimum HTTP::Proxy: > > $proxy = HTTP::Proxy->new( > port => 8080, > host => undef, > engine => 'Threaded', > ); > > $proxy->start; > > Client side: > export http_proxy=http://192.168.1.1:8080 > # curl -O http://mirror.utexas.edu/epel/6/x86_64/repodata/repomd.xml > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left > Speed > 101 4363 101 4363 0 0 14179 0 --:--:-- --:--:-- --:--:-- > 14211 > curl: (18) transfer closed with outstanding read data remaining > > > # curl --http1.0 -O > http://mirror.utexas.edu/epel/6/x86_64/repodata/repomd.xml > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left > Speed > 101 4363 101 4363 0 0 14826 0 --:--:-- --:--:-- --:--:-- > 14890 > > Note that with the --http1.0 option, the download succeeds. > > wget-based downloads work fine, but since yum is using libcurl and > pycurl as of RHEL6, yum no longer works with HTTP::Proxy. >

I used to have issues with apt-get under debian because it was using pipelining which HTTP::Proxy does not support. Debian have moved back from pipelining because that caused too much problems for too many people (it can still be enabled, but it's not the default). It's probably not the case here, as your example curl queries only fetch a single file at a time. Your test case is easy enough, so I've tried it with the latest version, and curl does not seem to fail any more. Can you confirm if you still have the issue? Regards, -- BooK

Tue Apr 21 16:44:59 2015 The RT System itself - Status changed from 'new' to 'open'

Mon Jun 15 16:36:27 2015 book [...] cpan.org - Correspondence added

RT-Send-CC:

JTOPJIAN [...] cpan.org

On Tue Apr 21 16:44:59 2015, BOOK wrote: Show quoted text

> On Mon Jul 11 18:15:33 2011, JTOPJIAN wrote: > > I used to have issues with apt-get under debian because it was using > pipelining > which HTTP::Proxy does not support. Debian have moved back from > pipelining because > that caused too much problems for too many people (it can still be > enabled, but > it's not the default). > > It's probably not the case here, as your example curl queries only > fetch a single > file at a time. > > Your test case is easy enough, so I've tried it with the latest > version, and > curl does not seem to fail any more. >

I've just tried again using explicitely the --http1.1 and --http1.0 options, and in both cases I got the whole file without error. Here's the curl version I used: $ curl --version curl 7.42.1 (x86_64-pc-linux-gnu) libcurl/7.42.1 GnuTLS/3.3.15 zlib/1.2.8 libidn/1.30 libssh2/1.5.0 librtmp/2.3 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP UnixSockets Show quoted text

> Can you confirm if you still have the issue? >

Thanks, -- BooK