Bug #61368 for WWW-Curl: Retrieving CURLINFO_CONTENT_TYPE leads to segdump on certain URLs

Wed Sep 15 12:48:19 2010 Lenny.Rachitsky [...] neustar.biz - Ticket created

Subject:	Retrieving CURLINFO_CONTENT_TYPE leads to segdump on certain URLs
Date:	Wed, 15 Sep 2010 12:48:01 -0400
To:	"bug-WWW-Curl [...] rt.cpan.org" <bug-WWW-Curl [...] rt.cpan.org>
From:	"Rachitsky, Lenny" <Lenny.Rachitsky [...] neustar.biz>

Curl version: 7.21.0 WWW:::Curl version: 4.11 The following code leads to a segdump: -------------------------------------------------------------------------- #!/usr/bin/perl use strict; use WWW::Curl::Easy; my $url = 'http://b.scorecardresearch.com/r?c2=6035748&d.c=gif&d.o=cnn3global&d.x=69861201&d.t=page&d.u=www.cnn.com'; my $curl = WWW::Curl::Easy->new() or print STDERR "Unable to create curl object"; $curl->setopt(CURLOPT_URL, $url); my $response = $curl->perform(); my $responseCode = $curl->getinfo(CURLINFO_RESPONSE_CODE -------------------------------------------------------------------------- I presume it has something to do with that URL and it's response. Lenny

Sat Oct 16 17:02:22 2010 szbalint [...] cpan.org - Correspondence added

On Wed Sep 15 12:48:19 2010, Lenny.Rachitsky@neustar.biz wrote: Show quoted text

> Curl version: 7.21.0 > WWW:::Curl version: 4.11 > > The following code leads to a segdump: > > -------------------------------------------------------------------------- > #!/usr/bin/perl > > use strict; > use WWW::Curl::Easy; > > my $url = >

'http://b.scorecardresearch.com/r?c2=6035748&d.c=gif&d.o=cnn3global&d.x=69861201&d.t=page&d.u=www.cnn.com'; Show quoted text

> > my $curl = WWW::Curl::Easy->new() or print STDERR "Unable to create > curl object"; > $curl->setopt(CURLOPT_URL, $url); > > my $response = $curl->perform(); > > my $responseCode = $curl->getinfo(CURLINFO_RESPONSE_CODE > -------------------------------------------------------------------------- > > I presume it has something to do with that URL and it's response. > > > Lenny

Hey, Looks like the code you pasted got truncated along the way. Also, there were a couple of changes since 4.11, could you try testing with 4.13 and attach the complete test case if is still occuring? Unfortunately I was unable to reproduce the segfault you're seeing so far. Regards, Bálint

Sat Oct 16 17:02:23 2010 The RT System itself - Status changed from 'new' to 'open'

Mon Oct 25 17:12:07 2010 Lenny.Rachitsky [...] neustar.biz - Correspondence added

Subject:	Re: [rt.cpan.org #61368] Retrieving CURLINFO_CONTENT_TYPE leads to segdump on certain URLs
Date:	Mon, 25 Oct 2010 17:11:52 -0400
To:	"bug-WWW-Curl [...] rt.cpan.org" <bug-WWW-Curl [...] rt.cpan.org>
From:	"Rachitsky, Lenny" <Lenny.Rachitsky [...] neustar.biz>

Just upgraded to 4.14 and still seeing the problem. The key line is the last one, that retrieves the content type. This crashes on various domains, including scorecardresearch.com. I presume those domains are sending back something totally wacky. -------------------------------------------------------------------------- #!/usr/bin/perl use strict; use WWW::Curl::Easy; my $url = 'http://b.scorecardresearch.com/r?c2=6035748&d.c=gif&d.o=cnn3global&d.x=69861201&d.t=page&d.u=www.cnn.com'; my $curl = WWW::Curl::Easy->new() or print STDERR "Unable to create curl object"; $curl->setopt(CURLOPT_URL, $url); my $response = $curl->perform(); my $responseCode = $curl->getinfo(CURLINFO_RESPONSE_CODE); my $contentType = $curl->getinfo(CURLINFO_CONTENT_TYPE); -------------------------------------------------------------------------- Hope this doesn't get truncated. If it does, here is the key line: my $contentType = $curl->getinfo(CURLINFO_CONTENT_TYPE); On 10/16/10 2:02 PM, "Balint Szilakszi via RT" <bug-WWW-Curl@rt.cpan.org> wrote: <URL: https://rt.cpan.org/Ticket/Display.html?id=61368 > On Wed Sep 15 12:48:19 2010, Lenny.Rachitsky@neustar.biz wrote: Show quoted text

> Curl version: 7.21.0 > WWW:::Curl version: 4.11 > > The following code leads to a segdump: > > -------------------------------------------------------------------------- > #!/usr/bin/perl > > use strict; > use WWW::Curl::Easy; > > my $url = >

'http://b.scorecardresearch.com/r?c2=6035748&d.c=gif&d.o=cnn3global&d.x=69861201&d.t=page&d.u=www.cnn.com'; Show quoted text

> > my $curl = WWW::Curl::Easy->new() or print STDERR "Unable to create > curl object"; > $curl->setopt(CURLOPT_URL, $url); > > my $response = $curl->perform(); > > my $responseCode = $curl->getinfo(CURLINFO_RESPONSE_CODE > -------------------------------------------------------------------------- > > I presume it has something to do with that URL and it's response. > > > Lenny

Hey, Looks like the code you pasted got truncated along the way. Also, there were a couple of changes since 4.11, could you try testing with 4.13 and attach the complete test case if is still occuring? Unfortunately I was unable to reproduce the segfault you're seeing so far. Regards, Bálint -- Lenny Rachitsky Neustar Webmetrics / Head of R&D 9444 Waples St., San Diego CA 92121 Office: +1.858.352.3164 / lenny.rachitsky@webmetrics.com / www.webmetrics.com

Mon Mar 21 04:23:51 2011 GNATYNA [...] cpan.org - Correspondence added

got segfaults too. [gnatyna@CentOS-55-64-minimal batch]$ perl -le 'use WWW::Curl::Easy; $c = WWW::Curl::Easy->new(); $c->setopt(CURLOPT_URL, "asdfasfasdfasdfadfaf.ru"); $c->perform(); $h = $c->getinfo (CURLINFO_CONTENT_TYPE); print $h' Segmentation fault (core dumped) [gnatyna@CentOS-55-64-minimal batch]$ gdb backtrace #0 0x0000003493079a30 in strlen () from /lib64/libc.so.6 #1 0x0000003634699330 in Perl_newSVpv () from /usr/lib64/perl5/5.8.8/ x86_64-linux-thread-multi/CORE/libperl.so #2 0x00002b6a1f5fe551 in XS_WWW__Curl__Easy_getinfo (my_perl=0x204a010, cv=<value optimized out>) at Curl.xs:893 #3 0x0000003634690a96 in Perl_pp_entersub () from /usr/lib64/ perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so #4 0x000000363468a33e in Perl_runops_standard () from /usr/lib64/ perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so #5 0x000000363463808a in perl_run () from /usr/lib64/perl5/5.8.8/ x86_64-linux-thread-multi/CORE/libperl.so #6 0x00000000004017bc in main () [gnatyna@CentOS-55-64-minimal batch]$ uname -a Linux CentOS-55-64-minimal 2.6.18-194.32.1.el5.centos.plus #1 SMP Wed Jan 5 18:13:47 EST 2011 x86_64 x86_64 x86_64 GNU/Linux curl 7.21.4 (x86_64-unknown-linux-gnu) libcurl/7.21.4 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 libssh2/1.2.7 Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp Features: IDN IPv6 Largefile NTLM SSL libz perl -le 'use WWW::Curl::Easy; print $WWW::Curl::Easy::VERSION' 4.15 hope it will help to detect bug. on i586 curl works fine.

Mon Mar 21 04:28:34 2011 GNATYNA [...] cpan.org - Correspondence added

forgot perl version. perl -V Summary of my perl5 (revision 5 version 8 subversion 8) configuration: Platform: osname=linux, osvers=2.6.18-53.el5, archname=x86_64-linux-thread- multi uname='linux builder10.centos.org 2.6.18-53.el5 #1 smp mon nov 12 02:14:55 est 2007 x86_64 x86_64 x86_64 gnulinux ' config_args='-des -Doptimize=-O2 -g -pipe -Wall -Wp,- D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer- size=4 -m64 -mtune=generic -Dversion=5.8.8 -Dmyhostname=localhost - Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. - Dinstallprefix=/usr -Dprefix=/usr -Dlibpth=/usr/local/lib64 /lib64 /usr/ lib64 -Dprivlib=/usr/lib/perl5/5.8.8 -Dsitelib=/usr/lib/perl5/ site_perl/5.8.8 -Dvendorlib=/usr/lib/perl5/vendor_perl/5.8.8 -Darchlib=/ usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi -Dsitearch=/usr/lib64/ perl5/site_perl/5.8.8/x86_64-linux-thread-multi -Dvendorarch=/usr/lib64/ perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi -Darchname=x86_64- linux-thread-multi -Dvendorprefix=/usr -Dsiteprefix=/usr -Duseshrplib - Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun - Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/ less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto - Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto - Ud_endservent_r_proto -Ud_setservent_r_proto -Dinc_version_list=5.8.7 5.8.6 5.8.5 -Dscriptdir=/usr/bin' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include - D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm', optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions - fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic', cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe - Wdeclaration-after-statement -I/usr/local/include -I/usr/include/gdbm' ccversion='', gccversion='4.1.2 20080704 (Red Hat 4.1.2-48)', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='gcc', ldflags ='' libpth=/usr/local/lib64 /lib64 /usr/lib64 libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread - lc perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc libc=, so=so, useshrplib=true, libperl=libperl.so gnulibc_version='2.5' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E - Wl,-rpath,/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE' cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall -Wp,- D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer- size=4 -m64 -mtune=generic' Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_PERLIO USE_REENTRANT_API Built under linux Compiled at Sep 28 2010 08:50:01 @INC: /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi /usr/lib/perl5/5.8.8

Mon Mar 21 05:48:08 2011 GNATYNA [...] cpan.org - Correspondence added

found quick solution for WWW::Curl version 4.15. CURLINFO_CONTENT_TYPE may set vchar to NULL; diff Curl.xs Curl.xs.new 892,893c892,896 < curl_easy_getinfo(self->curl, option, &vchar); < RETVAL = newSVpv(vchar,0); --- Show quoted text

> if(CURLE_OK == curl_easy_getinfo(self->curl, option,

&vchar)){ Show quoted text

> if(vchar != 0){ > RETVAL = newSVpv(vchar,0); > } > }

http://curl.haxx.se/libcurl/c/curl_easy_getinfo.html CURLINFO_CONTENT_TYPE Pass a pointer to a char pointer to receive the content-type of the downloaded object. This is the value read from the Content-Type: field. If you get NULL, it means that the server didn't send a valid Content- Type header or that the protocol used doesn't support this.