Bug #110833 for CPAN: ftp_proxy seems to be ignored

Fri Jan 01 16:46:51 2016 $_ = 'spro^^*%*^6ut# [...] &$%*c>#!^!#&!pan.org'; y/a-z.@//cd; print - Ticket created

Subject:

ftp_proxy seems to be ignored

I have the ftp_proxy configuration variable set to an HTTP URL. CPAN.pm still tries to fetch the files from an FTP address with Net::FTP, but Net::FTP does not support HTTP proxies, so it sits there doing nothing (due to local firewall restrictions), before finally giving me this message: Proxy authentication needed! And then I just press return at the two prompts, after which it precedes to use curl: Trying with /usr/bin/curl -L -f -s -S --netrc-optional -U ":" -x "http://1.2.3.4:5678" to get ftp://mirror.xmission.com/CPAN/authors/01mailrc.txt.gz which works. But it goes through this for every single file to be fetched, making it take a very long time to get anything done! ftp_proxy with an http:// prefix should override any use of Net::FTP.

Sat Jan 02 10:34:59 2016 $_ = 'spro^^*%*^6ut# [...] &$%*c>#!^!#&!pan.org'; y/a-z.@//cd; print - Correspondence added

On Fri Jan 01 16:46:51 2016, SPROUT wrote: Show quoted text

> I have the ftp_proxy configuration variable set to an HTTP URL. > CPAN.pm still tries to fetch the files from an FTP address with > Net::FTP, but Net::FTP does not support HTTP proxies, so it sits there > doing nothing (due to local firewall restrictions), before finally > giving me this message: > > Proxy authentication needed! > > And then I just press return at the two prompts, after which it > precedes to use curl: > > Trying with > /usr/bin/curl -L -f -s -S --netrc-optional -U ":" -x > "http://1.2.3.4:5678" > to get > ftp://mirror.xmission.com/CPAN/authors/01mailrc.txt.gz > > which works. > > But it goes through this for every single file to be fetched, making > it take a very long time to get anything done! ftp_proxy with an > http:// prefix should override any use of Net::FTP.

For now, I’m using Devel::Hide to hide Net::FTP when running the CPAN shell, like this: DEVEL_HIDE_VERBOSE=0 DEVEL_HIDE_PM=Net::FTP perl -MDevel::Hide -MCPAN -eshell And I have to set the proxy_user and proxy_pass variables to a true value to stop getting the prompts for every file fetched. Luckily the proxy, which doesn’t require credentials, doesn’t care about the bogus ones. Everything is going smoothly now. But these are just workarounds.

Sat Jan 02 10:34:59 2016 The RT System itself - Status changed from 'new' to 'open'

Fri Jul 08 21:28:05 2016 ANDK [...] cpan.org - Correspondence added

I think I have two solutions, one is attached patch. If I understand you correctly, Net::FTP does not work with an http proxy. This patch would prevent that Net::FTP would ever be tried when an ftp_proxy matches /^http:/. It has the risk that other users trip over it because exactly this setting works for them. Or when Net::FTP gets fixed. The other solution is to set the config variable dontload_list: Show quoted text

> o conf dontload_list push Net::FTP

This should have a similar effect as Devel::Hide. Please let me know what you think. If you find the patch works and is safe, I'd apply it and see how other users are affected.

Subject:

0001-Prevent-trying-Net-FTP-when-an-ftp_proxy-variable-is.patch

From ba8629ff6703ea473e83c39d9b8d6e3da030bef9 Mon Sep 17 00:00:00 2001 From: Andreas Koenig <andk@cpan.org> Date: Sat, 9 Jul 2016 03:04:25 +0200 Subject: [PATCH] Prevent trying Net::FTP when an ftp_proxy variable is set to an http proxy - reported in https://rt.cpan.org/Ticket/Display.html?id=110833 --- lib/CPAN.pm | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/lib/CPAN.pm b/lib/CPAN.pm index ab2d00f..6e460a8 100644 --- a/lib/CPAN.pm +++ b/lib/CPAN.pm @@ -1064,6 +1064,16 @@ sub has_usable { }, ], 'Net::FTP' => [ + sub { + my $var = $CPAN::Config->{ftp_proxy} || $ENV{ftp_proxy}; + if ($var and $var =~ /^http:/) { + # rt #110833 + for ("Net::FTP cannot handle http proxy") { + $CPAN::Frontend->mywarn($_); + die $_; + } + } + }, sub {require Net::FTP}, sub {require Net::Config}, ], -- 2.7.0

Sat Jul 09 10:06:17 2016 $_ = 'spro^^*%*^6ut# [...] &$%*c>#!^!#&!pan.org'; y/a-z.@//cd; print - Correspondence added

On Fri Jul 08 21:28:05 2016, ANDK wrote: Show quoted text

> I think I have two solutions, one is attached patch. If I understand > you correctly, Net::FTP does not work with an http proxy. > > This patch would prevent that Net::FTP would ever be tried when an > ftp_proxy matches /^http:/. It has the risk that other users trip over > it because exactly this setting works for them. Or when Net::FTP gets > fixed.

I do not think it is a bug per se if Net::FTP does not handle HTTP proxies. From the client’s point of view, *everything* is HTTP if you are going through an HTTP proxy, regardless of the URL prefix. ‘Fixing’ Net::FTP would ultimately entail having it fall back to LWP or HTTP::Tiny anyway, so I don’t think trying to use it to begin with even makes sense. Hence, I do not think it should affect other users. Show quoted text

> > The other solution is to set the config variable dontload_list: >

> > o conf dontload_list push Net::FTP

I should read the documentation. :-) Show quoted text

> > This should have a similar effect as Devel::Hide. > > Please let me know what you think. If you find the patch works and is > safe, I'd apply it and see how other users are affected.

The patch works for me. Thank you.

Sat Jul 09 12:59:21 2016 $_ = 'spro^^*%*^6ut# [...] &$%*c>#!^!#&!pan.org'; y/a-z.@//cd; print - Correspondence added

On Fri Jul 08 21:28:05 2016, ANDK wrote: Show quoted text

> I think I have two solutions, one is attached patch. If I understand > you correctly, Net::FTP does not work with an http proxy. > > This patch would prevent that Net::FTP would ever be tried when an > ftp_proxy matches /^http:/. It has the risk that other users trip over > it because exactly this setting works for them. Or when Net::FTP gets > fixed. > > The other solution is to set the config variable dontload_list: >

> > o conf dontload_list push Net::FTP

> > This should have a similar effect as Devel::Hide. > > Please let me know what you think. If you find the patch works and is > safe, I'd apply it and see how other users are affected.

One comment about the patch: URL schemes are case-tolerant, so the regexp should have /i.

Sun Jul 10 07:20:16 2016 ANDK [...] cpan.org - Correspondence added

Thanks, two commits are on the repository waiting for the next release. Setting status to `patched`.

Sun Jul 10 07:20:27 2016 ANDK [...] cpan.org - Status changed from 'open' to 'patched'

Mon Dec 24 15:54:18 2018 ANDK [...] cpan.org - Status changed from 'patched' to 'resolved'

Mon Dec 24 15:54:18 2018 ANDK [...] cpan.org - Fixed in 2.16 added