Bug #13485 for Net-SNMP: Corrupt messages with tainted variables

Thu Jun 30 12:43:04 2005 Guest - Ticket created

Subject:

Corrupt messages with tainted variables

I've just stumbled across a problem whereby if TAINTED variables is used in a varbind value, then the SNMP message appears to be corrupt (snmptrad cannot format). I run tests with a tainted variable, then overwriting it's value with a constant (untainted). The debug from Net:: SNMP show different hex messages and snmptrapd shows same diff AND fails to format the tainted version but works fine with untainted. I've searched docs and bugs for reference to taint. If this is a new default, I will collect detailed info and submit. Please advise if this is unknown.

Thu Jun 30 13:16:03 2005 dtown [...] cpan.org - Correspondence added 5 min

[guest - Thu Jun 30 12:43:04 2005]: Show quoted text

> I've just stumbled across a problem whereby if TAINTED variables > is used in a varbind value, then the SNMP message appears to be > corrupt (snmptrad cannot format). I run tests with a tainted variable, > then > overwriting it's value with a constant (untainted). The debug from > Net:: > SNMP show different hex messages and snmptrapd shows same diff > AND fails to format the tainted version but works fine with untainted. > > I've searched docs and bugs for reference to taint. If this is a new > default, I will collect detailed info and submit. Please advise if > this is > unknown.

Can you provide an example of what you are seeing? The distribution comes with a "snmpset.pl" script which takes command line arguments to populate the varBindList. Command line arguments are considered "tainted" by Perl. I have not seen problems with this in the past. What version of Perl and Net::SNMP are you using? -David

Thu Jun 30 13:16:04 2005 dtown [...] cpan.org - Given to DTOWN

Fri Jul 01 04:45:40 2005 Guest - Correspondence added

From:

Graeme V

Show quoted text

> Can you provide an example of what you are seeing? The distribution > comes with a "snmpset.pl" script which takes command line arguments to > populate the varBindList. Command line arguments are considered > "tainted" by Perl. I have not seen problems with this in the past.

...elided.. Show quoted text

> > -David

OK thanks David, this does not sound like a 'well known problem' , I'll try to produce a minimal working+failing version with logs. If I don't manage it in the next couple of days, It'll be a couple of weeks before I can get near a computer :-( .. I had two other developers sit with me and go line by line through the code, we ended up with things like if ($st eq 200) {print "value IS 200\n";} because we stopped trusting our eyes :-). The upside was, it was VERY repeatable.

Mon Jul 11 13:50:08 2005 dtown [...] cpan.org - Status changed from 'new' to 'open'

Mon Jul 11 13:50:08 2005 dtown [...] cpan.org - Correspondence added 5 min

[guest - Fri Jul 1 04:45:40 2005]: Show quoted text

> > OK thanks David, this does not sound like a 'well known problem' , > I'll try to produce a minimal working+failing version with logs. If I > don't manage it in the next couple of days, It'll be a couple of weeks > before I can get near a computer :-( .. I had two other developers sit > with me and go line by line through the code, we ended up with things > like if ($st eq 200) {print "value IS 200\n";} because we stopped > trusting our eyes :-). The upside was, it was VERY repeatable. > >

Are you sure that the problem is really tainted variables? Where are the values that you are placing into the varBindList coming from? Are they being read from some sort of database? An issue was reported in the past where values read from an Oracle database were being flagged as UTF8 (Unicode) characters. When displayed, they appeared correctly, but when encoded they included the extended bytes. -David

Mon Aug 01 11:09:31 2005 Guest - Correspondence added

From:

Graeme vetterlein

Show quoted text

> Are you sure that the problem is really tainted variables? Where are > the values that you are placing into the varBindList coming from?

I just tried to create a cut-down example by getting 'tainted' values from LWP (the retuned content from a request) ... this is similar to my real case, however it DID NOT exhibit the problem. To be sure that has not gone away I reran various cut down versions of the original code. The orginal code sends XML requests (via HTTP) the uses the XML parsing modules to extract status&codes. The number in question is a code comming back via that route. The change (in original code) I make in the 'client' is: Original: $pairs{$tagname} = $text; With untainting (at least that whit I intended it to do :-) $text =~ /(.*)/; $pairs{$tagname} = $1; The 'untainted version' produces reasonable looking output from snmptrapd(1) run as snmptrapd -d -f -P -DALL. The 'tainted version' produces just the hex dump (no formatted version) AND a message (from debug?) saying sess_process_packet: 'parse fail' ... I'm still trying to produce a cut-down version, but it's proving more difficult than I expected

Tue Aug 02 04:17:41 2005 Guest - Correspondence added

From:

Graeme vetterlein

Show quoted text

> Are you sure that the problem is really tainted variables? Where are > the values that you are placing into the varBindList coming from?

I just tried to create a cut-down example by getting 'tainted' values from LWP (the retuned content from a request) ... this is similar to my real case, however it DID NOT exhibit the problem. To be sure that has not gone away I reran various cut down versions of the original code. The orginal code sends XML requests (via HTTP) the uses the XML parsing modules to extract status&codes. The number in question is a code comming back via that route. The change (in original code) I make in the 'client' is: Original: $pairs{$tagname} = $text; With untainting (at least that whit I intended it to do :-) $text =~ /(.*)/; $pairs{$tagname} = $1; The 'untainted version' produces reasonable looking output from snmptrapd(1) run as snmptrapd -d -f -P -DALL. The 'tainted version' produces just the hex dump (no formatted version) AND a message (from debug?) saying sess_process_packet: 'parse fail' ... I'm still trying to produce a cut-down version, but it's proving more difficult than I expected

Wed Aug 03 15:25:39 2005 dtown [...] cpan.org - Correspondence added 10 min

You never replied with the version of Perl and Net::SNMP. But from what I can reproduce, you are probably running Perl 5.6.1 and Net::SNMP prior to 5.0.0. The handling of UFT8 strings in Perl 5.6.1 was not really ready for prime time. A prepend call to substr() in Perl 5.6.1 with a UFT8 string would result in "garbage" being added to the string. When the Net::SNMP module builds a packet it uses prepend calls to substr(). The "untainting" example you provided actually "changes" the string from a UFT8 string to a regular string (in 5.6.1). You can also add the line "use bytes;" to your script which tells Perl to treat all strings as plain ASCII characters. This line was added to Net::SNMP in v5.0.0. -David [guest - Tue Aug 2 04:17:41 2005]: Show quoted text

>

> > Are you sure that the problem is really tainted variables? Where are > > the values that you are placing into the varBindList coming from?

> > I just tried to create a cut-down example by getting 'tainted' values > from LWP (the retuned content from a request) ... this is similar to my > real case, however it DID NOT exhibit the problem. To be sure that has > not gone away I reran various cut down versions of the original code. > > The orginal code sends XML requests (via HTTP) the uses the XML parsing > modules to extract status&codes. The number in question is a code > comming back via that route. > > The change (in original code) I make in the 'client' is: > > Original: > $pairs{$tagname} = $text; > > > With untainting (at least that whit I intended it to do :-) > $text =~ /(.*)/; > $pairs{$tagname} = $1; > > The 'untainted version' produces reasonable looking output from > snmptrapd(1) run as snmptrapd -d -f -P -DALL. > > The 'tainted version' > produces just the hex dump (no formatted version) AND a message (from > debug?) saying sess_process_packet: 'parse fail' > > ... I'm still trying to produce a cut-down version, but it's proving > more difficult than I expected

Wed Aug 03 15:25:40 2005 dtown [...] cpan.org - Status changed from 'open' to 'resolved'

Thu Aug 04 08:52:24 2005 Guest - Correspondence added

[DTOWN - Wed Aug 3 15:25:39 2005]: Show quoted text

> You never replied with the version of Perl and Net::SNMP. But from

what Show quoted text

> I can reproduce, you are probably running Perl 5.6.1 and Net::SNMP

prior Show quoted text

> to 5.0.0.

Sorry about that. While I thought this was a new problem, rather than something well-known, I was planning to produce a cut-down test case with all the details of environment etc. However, I suspect you're correct about the UTF-8 :-) ... To that end, here are the versions: /data/perl/bin/perl -v This is perl, v5.6.1 built for sun4-solaris Copyright 1987-2001, Larry Wall .... # -*- mode: perl -*- # ======================================================================== ==== package Net::SNMP; # $Id: SNMP.pm,v 4.8 2003/09/11 19:14:42 dtown Exp $ # Copyright (c) 1998-2003 David M. Town <dtown@cpan.org> # All rights reserved. Show quoted text

> The handling of UFT8 strings in Perl 5.6.1 was not really ready for > prime time. A prepend call to substr() in Perl 5.6.1 with a UFT8

string Show quoted text

> would result in "garbage" being added to the string. When the > Net::SNMP module builds a packet it uses prepend calls to substr().

I was not aware that any of the data I was getting was UTF-8 (I am now!). The XML requests I'm generating are going to a new 'Box' and there is scant little documentation about the return codes etc ... mind you, had they said it was UTF-8, I might well have missed it :-) ! Show quoted text

> The "untainting" example you provided actually "changes" the string

from Show quoted text

> a UFT8 string to a regular string (in 5.6.1).

Ahh! ... as a matter of interest, how could I have found that out (without you telling me)? ... I guess this is new 'side-effect' (in X3J11 sense) of RE that started happening with utf-8 support? Show quoted text

> You can also add the line > "use bytes;" to your script which tells Perl to treat all strings as > plain ASCII characters. This line was added to Net::SNMP in v5.0.0.

Will that help? The XML service IS (I now realise) sending me UTF-8 codes. Now in this example it is in the 'ASCII range' but they also send textual error messages, which I guess might include some those new new (horrrible) quotes. I did also consider using UTF8STRING on varbinds but I guess from what you're saying that won't help. I guess I need to CONVERT from UTF-8 to say 8859-1. This is sorta what you're saying the 'untainting' bit does ... so what would happen if that untainting code met a non-ascii char? . Thanks for all you help with, especially as it turns out to be my screw-up :-)

Thu Aug 04 08:52:25 2005 The RT System itself - Status changed from 'resolved' to 'open'

Thu Aug 04 13:51:57 2005 dtown [...] cpan.org - Correspondence added 5 min

[guest - Thu Aug 4 08:52:24 2005]: Show quoted text

> > The "untainting" example you provided actually "changes" the string

> from

> > a UFT8 string to a regular string (in 5.6.1).

> > Ahh! ... as a matter of interest, how could I have found that out > (without you telling me)? ... I guess this is new 'side-effect' (in > X3J11 sense) of RE that started happening with utf-8 support?

I had to use the module "Devel::Peek" and call the function Dump() on the variable in question. The handling of UNICODE is supposed to be transparent, so there is no easy way to determine this information. Show quoted text

>

> > You can also add the line > > "use bytes;" to your script which tells Perl to treat all strings as > > plain ASCII characters. This line was added to Net::SNMP in v5.0.0.

> > Will that help? The XML service IS (I now realise) sending me UTF-8 > codes. Now in this example it is in the 'ASCII range' but they also > send textual error messages, which I guess might include some those new > new (horrrible) quotes.

What is most likely happening is that the variables populated by the XML library contained only ASCII character, but the variable is being flag as UFT8. The substr() function in 5.6.1 does not handle variables flagged as UFT8 properly. Show quoted text

> I did also consider using UTF8STRING on varbinds > but I guess from what you're saying that won't help. > > I guess I need to CONVERT from UTF-8 to say 8859-1. This is sorta what > you're saying the 'untainting' bit does ... so what would happen if that > untainting code met a non-ascii char? > > .

Your best bet is to use Net::SNMP v5.1.0 and at least Perl 5.8.0. Putting "use bytes;" in your script will remove any UFT8 characters. The "use bytes;" in the Net::SNMP module will take any UFT8 variable and transmit as UFT8 encoded bytes. -David

Thu Aug 04 13:51:58 2005 dtown [...] cpan.org - Status changed from 'open' to 'resolved'

Thu Aug 04 13:53:38 2005 dtown [...] cpan.org - Fixed in 5.0.0 added

Thu Aug 04 13:53:38 2005 dtown [...] cpan.org - Broken in 4.1.2 added