| Subject: | Inefficient use of network resources |
| Date: | Sun, 08 Jan 2012 16:51:00 +0200 |
| To: | bug-Net-SNMP-Vendor [...] rt.cpan.org |
| From: | Dave Stafford <Dave.Stafford [...] globis.net> |
Hi,
this is a useful idea for some network management work i'm doing,
but the documentation is incorrect, and the module itself is very
inefficient in terms of network usage. Given that I'm located
somewhere that the Internet is very slow, and that the IANA file is
quite large (3.1 Mbytes), it is unusable as a tool for me.
Problems:
First problem is that the documentation is incorrect, sysid should be sysoid
my $vendor = $v->lookup(sysid => $sysObjectID);
^
BTW I cannot really see the point of making the user pass a hash to
the lookup routine. It only does one thing, which is lookup a
sysObjectID. Wouldn't it be much easier, and a *lot* less error prone
to simply pass the sysObjectID itself? ie:
my $vendor = $v->lookup($sysObjectID);
Secondly the usage is very slow and network inefficient.
The IANA file is downloaded everytime just to check the last modified date.
The download does not use compression
You can use the If-modified-since HTTP header, and the file will only
be downloaded if it has changed since your last download date. You can
also use compression to reduce the size of the network download.
An additional small efficiency is to only load the cached data if the
online file is not downloaded.
For example (in partial code):
use strict;
use HTTP::Date;
use LWP::UserAgent;
my $file="sysoid.db";
my $url='http://www.iana.org/assignments/enterprise-numbers';
my $ua = LWP::UserAgent->new;
my $can_accept = HTTP::Message::decodable; # Allow compression
my $lastChecked = (stat($file))[9]; # Get last modify
time of local db
my $chkDate= HTTP::Date::time2str($lastChecked); # convert to HTML
usable format
# now add headers to the request
$ua->default_header('If-Modified-Since' => $chkDate, 'Accept-Encoding'
=> $can_accept);
my $response = $ua->get($url);
## Data has not changed so load the cached data
if ($response->code == 304) {
$_sysoid_cache=load_cached()
}
## if the data has changed, download the file, parse it, save
## to the loacal cache, and return the data object
elsif ($response->is_success) {
$_sysoid_cache=parse_iana( $response->decoded_content );
}
## if some error, print an error (or not) and load the cached data
else {
print "Error " . $response->code . ": '" . $response->status_line . "'\n";
$_sysoid_cache=load_cached()
}
The process is very slow on my PC and the DBM::Deep does not seem to
work well at all. It produces a huge file (> 36 mbytes) - perhaps it
does not work well on windows 7. However, if I change DBM::Deep for
Storable it works fine on my system. The data file is then about 6
Mbytes.
Another small point, your parsing of the sysObjectID is too strict in
the method lookup(). Some systems will return the string without the
leading dot, ie 1.3.6.1.4.1.14988.1
Make the first dot optional and change
/^(\.1\.3\.6\.1\.4\.1\.)?(\d+)/;
to
/^(\.*1\.3\.6\.1\.4\.1\.)?(\d+)/;
I hope this feedback helps. The idea is good, and I would definitely
use it (if it worked;-)
regards,
Dave