Skip Menu |

This queue is for tickets about the Weather-NOAA-Alert CPAN distribution.

Report information
The Basics
Id: 75485
Status: open
Priority: 0/
Queue: Weather-NOAA-Alert

People
Owner: STOVENOUR [...] cpan.org
Requestors: ryan [...] bebeau.org
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: Adding in addition XML elements including nested elements
Date: Fri, 02 Mar 2012 14:34:36 -0500
To: bug-Weather-NOAA-Alert [...] rt.cpan.org
From: Ryan Bebeau <ryan [...] bebeau.org>
Hello, First of all, wanted to thank you for writing this module. Second, I've included a patch to allow the nested elements of the CAP to be stored in the events hash. I also have code that would allow someone to know if a latitude/longitude is contained within a polygon. That should help to determine if a storm based warning surrounds a specific area smaller than a county. Included is the patch I made, hopefully it can be helpful. If you have further questions, please let me know. Thanks, Ryan Bebeau

Message body is not shown because sender requested not to inline it.

On Fri Mar 02 14:34:51 2012, ryan@bebeau.org wrote: Show quoted text
> Hello, > > First of all, wanted to thank you for writing this module.
I'm glad that others can use this module. I spent many hours reverse engineering the poorly documented interface from NOAA. Show quoted text
> Second, I've included a patch to allow the nested elements of the CAP to > be stored in the events hash. I also have code that would allow someone > to know if a latitude/longitude is contained within a polygon. > That should help to determine if a storm based warning surrounds a > specific area smaller than a county.
:) I also added code to support point in polygon detection. I took a slightly different approach. My approach was first to separate the "polled" counties from "detected" counties. I then added support for detection points. With this method the module user can "arm" counties and specific points for alert detection. The cap data structure then indicates if a tracked alert is inDetectionZone. The nice thing about this approach is that the module user can add multiple detection points and the module takes care of iterating over each point. Show quoted text
> Included is the patch I made, hopefully it can be helpful.
I would like to combine what you and I have done. 1st a couple of questions. I implemented my own point in polygon code. Do you think that Math::Polygon will be better? Maybe mathematically there is an advantage to the existing module? I don't mind using it if there is a good reason but otherwise I'd like to limit the number of dependencies. 2nd you've added support for nested elements. Is this only to get at the polygon? Or do you think there is a general need? My updates retrieve the polygon data. Show quoted text
> If you have further questions, please let me know.
I really appreciate the time you've taken to use and modify the code. I am truly interested in combining the best of our ideas. Let me know what you think. I've attached my updated Alert.pm, example.pl, and noaa_alert_win32.pl for you to take a look at. Show quoted text
> Thanks, > > Ryan Bebeau
Subject: noaa_alert_win32-diff.txt
--- Weather-NOAA-Alert-0.90/examples/noaa_alert_win32.pl 2011-05-01 13:26:48.421875000 -0500 +++ Weather-NOAA-Alert/examples/noaa_alert_win32.pl 2011-05-02 01:57:31.687500000 -0500 @@ -1,6 +1,6 @@ use strict; -#use lib '../Weather-NOAA-Alert/lib'; + use Weather::NOAA::Alert; use Win32::OLE; @@ -9,15 +9,16 @@ use Data::Dumper; -our $VERSION = '2.10'; +our $VERSION = '2.20'; -#V2.00 - Major rewrite to accomodate CAP 1.1 format. Moved CAP parsing to +#V2.00 - Major rewrite to accommodate CAP 1.1 format. Moved CAP parsing to # the Weather-NOAA-Alert module #V2.10 - Add tracking for the same CAP event in multiple Zones. The same -# alert can show up in multiple zones when polling for ajacent zones. For +# alert can show up in multiple zones when polling for adjacent zones. For # example, polling for both Dallas and Collin counties in Texas might produce # the same CAP tornado watch event. Code will now only schedule actions for # the first instance. +#V2.20 - Add support for detectionPoints and detectionZones #XXX-Add an audio test mode where the first message is always generated @@ -47,7 +48,7 @@ #between the urgency category and the alerts. For example from simply #looking over the current data it appears that flood warnings are #"Expected" while severe thunderstorm warnings are "Immediate". I might -#write a data coorelator that pulls the data a couple times a day and +#write a data correlator that pulls the data a couple times a day and #builds a table of values for each even type. #Why not just build as many queues as we have urgency information?? @@ -95,6 +96,10 @@ #Create a NOAA Alert Object, configure it, and get a reference to the events hash our $alert = Weather::NOAA::Alert->new(\@zones); +$alert->detectionZones(['TXC085', 'TXZ104']); +$alert->detectionPoints([[33.090394, -96.753218]]); +$alert->formatTime(1); +$alert->formatAsterisk(1); $alert->printLog(1); $alert->printActions(1); $alert->errorLog(1); @@ -272,7 +277,7 @@ $playAlertWav = $eventActions{$event}[2]; } else { #some suitable defaults if a new product springs to life - #Speek the text but don't play an alert tone + #Speak the text but don't play an alert tone $eventRepeat = $REPEAT_SECS; $includeWOthers = 0; $playAlertWav = 0; @@ -312,7 +317,7 @@ # ->{'playAlertWav'}=0 # ->{'text'}='text to speak' - #May need to add some colums to %eventActions so that different + #May need to add some columns to %eventActions so that different # action rules can apply to TTS vs. xAPAlert #For now xAPAlert is not implemented @@ -325,7 +330,12 @@ #Don't queue the action if this CAP id has already been #processed in a different zone. (e.g. same watch in two counties) - if( !exists($capIdSeen{$capId})) { + if( exists($capIdSeen{$capId})) { + print "Skip dup -> $event\n"; + } elsif( not $events->{$zone}{$capId}{'inDetectionZone'}) { + $capIdSeen{$capId} = 1; + print "not inDetectionZone -> $event\n"; + } else { $capIdSeen{$capId} = 1; print "Queuing -> $event\n"; push( @{$actionQueues{$severity}}, @@ -337,8 +347,6 @@ } ); $events->{$zone}{$capId}{'actionTime'} = time; - } else { - print "Skip dup -> $event\n"; } } #foreach my $capId
Subject: example-diff.txt
--- Weather-NOAA-Alert-0.90/examples/example.pl 2011-05-01 13:26:48.421875000 -0500 +++ Weather-NOAA-Alert/examples/example.pl 2011-05-02 00:44:39.718070500 -0500 @@ -1,10 +1,14 @@ +use strict; use Weather::NOAA::Alert; use Data::Dumper; +$Data::Dumper::Indent = 1; my $SLEEP_SECS = 60; #Poll every 1 minute #my $alert = Weather::NOAA::Alert->new(['US']); my $alert = Weather::NOAA::Alert->new(['TXC085', 'TXZ097']); +$alert->detectionZones(['TXC085']); +$alert->detectionPoints([[33.090394, -96.753218]]); $alert->printLog(1); $alert->errorLog(1);
Subject: Alert-diff.txt
--- Weather-NOAA-Alert-0.90/lib/Weather/NOAA/Alert.pm 2011-05-01 13:26:48.437500000 -0500 +++ Weather-NOAA-Alert/lib/Weather/NOAA/Alert.pm 2011-05-02 02:20:59.934619600 -0500 @@ -19,11 +19,11 @@ =head1 VERSION -Version 0.90 +Version 0.91 =cut -our $VERSION = '0.90'; +our $VERSION = '0.91'; =head1 SYNOPSIS @@ -49,16 +49,38 @@ =item * Zone Maps: L<http://www.weather.gov/mirs/public/prods/maps/pfzones_list.htm> =back + +=head 2 Affected Area + +The affected area for NOAA NWS watches, warnings, and advisories fall into +one of three categories. They are either in a forecast zone, forecast county, +or within a polygon defined in the alert. An alert is considered to affect the +entire forecast zone or county if the alert is supplied in the respective zone +or county file and the C<E<lt>areaE<gt>E<lt>polygonE<gt>> enclosure is empty. +If an C<E<lt>areaE<gt>E<lt>polygonE<gt>> enclosure is supplied then the alert +only applies to the area enclosed by the C<E<lt>areaE<gt>E<lt>polygonE<gt>>. + +Weather::NOAA::Alert can be armed with zones and latitude/longitude points which +are compared to the alert area. The C<inDetectionZone> flag will be set in +the C<events> hash if one or more of the zones and/or points are within the +alert area. This permits monitoring other counties and reacting differently +for alerts within the detection are vs. alerts outside the detection area. In +the simplest case set the same zones for polling as for detection and provide a +latitude/longitude point for polygon based alerts. + =head1 EXAMPLE use Weather::NOAA::Alert; use Data::Dumper; - + $Data::Dumper::Indent = 1; + my $SLEEP_SECS = 60; #Poll every 1 minute #my $alert = Weather::NOAA::Alert->new(['US']); my $alert = Weather::NOAA::Alert->new(['TXC085', 'TXZ097']); + $alert->detectionZones(['TXC085']); + $alert->detectionPoints([[33.090394, -96.753218]]); $alert->printLog(1); $alert->errorLog(1); @@ -99,6 +121,8 @@ my $class = shift; my $self = {}; + $self->{detectionPoints} = []; + $self->{detectionZones} = []; $self->{events} = {}; $self->{formatTime} = 0; $self->{formatAsterisk} = 0; @@ -122,19 +146,53 @@ return $self; } -=head2 B<zones> - Set the monitored zone list +=head2 B<zones> - Set the polled zone list $object->zones([zone1, zone2, ...]); @zones = $object->zones(); -Setting the zone list will overwrite the existing list with the -supplied list. If called with no arguments, returns a reference -to the current zone array. To return data for all zones use "US". +List of zones whose ATOM files will be polled and monitored for +new alerts. Setting the zone list will overwrite the existing list +with the supplied list. If called with no arguments, returns a +reference to the current zone array. To return data for all zones +use "US". =cut sub zones { $_[0]->{zones}=$_[1] if defined $_[1]; return $_[0]->{zones}; } +=head2 B<detectionZones> - Set the detection zone list + + $object->detectionZones([zone1, zone2, ...]); + @zones = $object->detectionZones(); + +Events for zones in the C<detectionZone> list will set the +C<inDetectionZone> flag for the event in the events hash unless the +event contains a non-empty C<E<lt>areaE<gt>E<lt>polygonE<gt>> +enclosure. Setting the zone list will overwrite the existing list +with the supplied list. If called with no arguments, returns a +reference to the current C<detectionZone> array. + +=cut + +sub detectionZones { $_[0]->{detectionZones}=$_[1] if defined $_[1]; return $_[0]->{detectionZones}; } + +=head2 B<detectionPoints> - Set the detection point list + + $object->detectionPoints([[lat1, lon1], [lat2, lon2], ...]); + @zones = $object->detectionPoints(); + +Events that contain a non-empty C<E<lt>areaE<gt>E<lt>polygonE<gt>> +enclosure and where one or more detectionPoints are within the polygon +will set the C<inDetectionZone> flag for the event in the events hash. +Setting the C<detectionPoints> list will overwrite the existing +list with the supplied list. If called with no arguments, returns +a reference to the current C<detectionPoints> array. + +=cut + +sub detectionPoints { $_[0]->{detectionPoints}=$_[1] if defined $_[1]; return $_[0]->{detectionPoints}; } + =head2 B<get_events> - get a reference to the alert object events hash %events = $object->get_events(); @@ -158,6 +216,7 @@ C< -E<gt>{'effective'}> C< -E<gt>{'headline'}> C< -E<gt>{'expires'}> + C< -E<gt>{'inDetectionZone'}> Note that the hash keys are dynamically created from the <info> section of the event. If NOAA adds, renames, or removes an XML parameter it will also @@ -419,13 +478,13 @@ my ($self, $zone, $capId) = @_; my $capTwig= new XML::Twig( - TwigRoots => {'info' => 1}, -# TwigHandlers => {'info' => \&capInfoHandler}, - TwigHandlers => {'info' => sub { capInfoHandler( $self->{formatTime}, $self->{formatAsterisk}, @_) } }, + TwigRoots => {'/alert/info' => 1}, + TwigHandlers => {'/alert/info' => sub { capInfoHandler( $self->{formatTime}, $self->{formatAsterisk}, @_) } }, pretty_print => 'indented', ); - + my $capContent; + my $polygon = ''; if ($capContent = get($capId)) { if( defined( $self->{diagFile}) and $self->{diagDump}) { @@ -438,11 +497,32 @@ #Parse only the first <info> enclosure #Loop through it appending items to the event hash foreach my $child ($capTwig->root->first_child->children) { - #ignore nested items: eventCode, parameter, area; they're too hard :^) + #ignore nested items: eventCode, parameter; they're too hard :^) if($child->tag ne 'eventCode' and $child->tag ne 'parameter' and $child->tag ne 'area') { $self->{events}->{$zone}{$capId}{$child->tag} = $child->text; + } elsif( $child->tag eq 'area') { + foreach my $childArea ($child->children) { + $polygon = $childArea->text if($childArea->tag eq 'polygon'); + } + } + } + #check for inDetectionZone + #if there was a polygon check all the detectionPoints else + #search for the currently polled zone in detectionZones + if($polygon eq '') { + $self->{events}->{$zone}{$capId}{'inDetectionZone'} = + grep(/\b$zone\b/,@{$self->{detectionZones}}); + } else { + $self->{events}->{$zone}{$capId}{'polygon'} = $polygon; + $self->{events}->{$zone}{$capId}{'inDetectionZone'} = 0; + foreach my $point (@{$self->{detectionPoints}}) { + my @polyPoints = map { [split(/,/)] } split(/ /, $polygon); + if(pointInPolygon( \@polyPoints, $point)) { + $self->{events}->{$zone}{$capId}{'inDetectionZone'} = 1; + } } } + $self->{events}->{$zone}{$capId}{'delete'} = 0; } else { @@ -495,6 +575,7 @@ #except the description and instruction; they need other adjustments. my @children = $capInfo->children; foreach my $child (@children) { + next if($child->tag eq 'area'); my $childText = $child->text; if ($child->tag ne 'description' and $child->tag ne 'instruction') { #Adjust the formatting a little. Why would a CAP file @@ -505,13 +586,13 @@ $childText =~ s/\s+$//; #remove trailing spaces } else { if( $formatTime) { -# if( 1 ) { #Try to add colons to all the time fields. This allows #the MS SAPI engine to correctly pronounce the time - $childText =~ s/(\d{1,2}?)(\d{2})\s{1}(AM|PM)\s{1}[A-Z]{3}/$1:$2 $3/g; + $childText =~ s/(\d{1,2}?)(\d{2})\s{1}(AM|PM)\s{1}[A-Z]{1}[D|S]T/$1:$2 $3/g; + $childText =~ s/(\d{1,2}?)(\d{2})\s{1}(AM|PM)\s*\n/$1:$2 $3\n/g; + $childText =~ s/(\d{1,2}?)(\d{2})\s{1}(AM|PM)\s/$1:$2 $3 /g; } if( $formatAsterisk) { -# if( 1 ) { #Remove any "*" because it sounds real funny when SAPI #pronounces "asterisk" in the middle of the speech stream $childText =~ s/\*/ /g; @@ -521,6 +602,32 @@ } } +sub pointInPolygon { + my ($polygon, $point) = @_; + my $lat = $point->[0]; + my $lon = $point->[1]; + + my $i; + my $j = (scalar @$polygon) - 1; + my $inPoly = 0; + + #look for an odd number of intesecting points with line through point and + #the boundary of the polygon + for ( $i = 0; $i < scalar @$polygon; $i++) { + if ($polygon->[$i][1] < $lon && $polygon->[$j][1] >= $lon + || $polygon->[$j][1] < $lon && $polygon->[$i][1] >= $lon) { + if ($polygon->[$i][0] + ( $lon - $polygon->[$i][1]) / + ($polygon->[$j][1] - $polygon->[$i][1]) * ($polygon->[$j][0] + - $polygon->[$i][0]) < $lat) { + $inPoly = !$inPoly; + } + } + $j = $i; + } + return $inPoly; +} + + =head1 NOAA CAP CHALLENGES The following items represent fragile parts of the code caused by ambiguous @@ -561,22 +668,6 @@ =back -=head1 TODO - -=over 4 - -=item * B<Implement point detection based on latitude / longitude> - -Events contain a C<E<lt>polygonE<gt>E<lt>/polygonE<gt>> that could be -used to determine if a specific latitude / longitude is in the event -area. This would greatly reduce the number of alerts for a specific -point in a county since the NWS has recently started issuing and -expiring events for specific areas that are not bound by county. -Tracking at the county level can trigger several events over a short -period as a storm progresses through the county. - -=back - =head1 SEE ALSO =over 4
Subject: Re: [rt.cpan.org #75485] Adding in addition XML elements including nested elements
Date: Tue, 06 Mar 2012 12:17:35 -0500
To: bug-Weather-NOAA-Alert [...] rt.cpan.org
From: Ryan Bebeau <ryan [...] bebeau.org>
On 03/06/12 00:34, Michael Stovenour via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=75485> > > On Fri Mar 02 14:34:51 2012, ryan@bebeau.org wrote:
>> Hello, >> >> First of all, wanted to thank you for writing this module.
> I'm glad that others can use this module. I spent many hours reverse > engineering the poorly documented interface from NOAA. >
>> Second, I've included a patch to allow the nested elements of the CAP to >> be stored in the events hash. I also have code that would allow someone >> to know if a latitude/longitude is contained within a polygon. >> That should help to determine if a storm based warning surrounds a >> specific area smaller than a county.
> :) I also added code to support point in polygon detection. I took a > slightly different approach. My approach was first to separate the > "polled" counties from "detected" counties. I then added support for > detection points. With this method the module user can "arm" counties > and specific points for alert detection. The cap data structure then > indicates if a tracked alert is inDetectionZone. The nice thing about > this approach is that the module user can add multiple detection points > and the module takes care of iterating over each point. >
>> Included is the patch I made, hopefully it can be helpful.
> I would like to combine what you and I have done. 1st a couple of > questions. I implemented my own point in polygon code. Do you think > that Math::Polygon will be better? Maybe mathematically there is an > advantage to the existing module? I don't mind using it if there is a > good reason but otherwise I'd like to limit the number of dependencies. > 2nd you've added support for nested elements. Is this only to get at > the polygon? Or do you think there is a general need? My updates > retrieve the polygon data.
As far as detection of a point within the polygon, I just used Math::Polygon because I was lazy. Since only the 'contains' function is ever really called, it's probably a waste of resources to include Math::Polygon. For the nested elements, I do use at least the VTEC parameter if it is defined. The reason I do this is so that I can know if I've already acted on this alert before, as well as only acting on specific types of 'actions', for example, only alert on 'NEW' VTEC events, not CON (event continued) or EXT (extended time)/etc. The table I referenced for VTEC explanation is here: http://www.nws.noaa.gov/os/vtec/pdfs/VTEC_explanation6.pdf One other thing I noticed in your code is that if I have, let's say, 5 detection points in one county and I want to only alert when any detection point is covered by a polygon, I will not know which of the 5 is actually within the polygon when I check the events hash. My end goal is to make it so, for example, 5 different buildings are monitored and if building 1 is affected, then send an alert to contact #1, if building 2 is affected, send to a different contact, and so on. If you would include the matched detectionPoints in the events hash, that would eliminate that issue. It might possibly be helpful to include links in the Perl documentation for the end user to look up their coordinates. I've been using these: http://geocoder.us/ http://itouchmap.com/latlong.html I've never seen these things yet, but there is the <circle> tag defined in the CAP 1.1 document... pretty sure that is just for earthquake kinds of events, but I could possibly see it being used for a hurricane type of situation as well. Anyway, thanks again for your time in writing this. I wrote something similar to this a few years ago, but never ended up having the time to throw it up onto CPAN. I was pleasantly surprised when I found your module there. Thanks again. Ryan Bebeau Show quoted text
>> If you have further questions, please let me know.
> I really appreciate the time you've taken to use and modify the code. I > am truly interested in combining the best of our ideas. Let me know > what you think. I've attached my updated Alert.pm, example.pl, and > noaa_alert_win32.pl for you to take a look at. >
>> Thanks, >> >> Ryan Bebeau
>
Any chances on a new release with these included? I was just dumb and wasted a lot of time working on a patch to enable me doing the polygon stuff in my code. It's my fault for not checking the bug tracker first, of course. here are my two changes so I don't feel like I completely wasted my time, the approach is slightly different. From 0066499a91cc317587d5de54e71f457ad7c05194 Mon Sep 17 00:00:00 2001 From: mikegrb <mgreb@linode.com> Date: Sun, 28 Jul 2013 23:32:37 -0400 Subject: [PATCH 1/2] don't reformat the text of elements with child elements --- lib/Weather/NOAA/Alert.pm | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/Weather/NOAA/Alert.pm b/lib/Weather/NOAA/Alert.pm index 1ac0ed7..d376291 100755 --- a/lib/Weather/NOAA/Alert.pm +++ b/lib/Weather/NOAA/Alert.pm @@ -495,6 +495,7 @@ sub capFormatTags { #except the description and instruction; they need other adjustments. my @children = $capInfo->children; foreach my $child (@children) { + next if $child->children; my $childText = $child->text; if ($child->tag ne 'description' and $child->tag ne 'instruction') { #Adjust the formatting a little. Why would a CAP file -- 1.8.1.6 From b515a234641d963e67b369175966c3246c203433 Mon Sep 17 00:00:00 2001 From: mikegrb <mgreb@linode.com> Date: Sun, 28 Jul 2013 23:32:59 -0400 Subject: [PATCH 2/2] add the area polygon to the CAP event data structure --- lib/Weather/NOAA/Alert.pm | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/Weather/NOAA/Alert.pm b/lib/Weather/NOAA/Alert.pm index d376291..fa38e94 100755 --- a/lib/Weather/NOAA/Alert.pm +++ b/lib/Weather/NOAA/Alert.pm @@ -443,6 +443,8 @@ sub retrieveCAP { $self->{events}->{$zone}{$capId}{$child->tag} = $child->text; } } + $self->{events}->{$zone}{$capId}{polygon} + = $capTwig->findvalue('//info/area/polygon'); $self->{events}->{$zone}{$capId}{'delete'} = 0; } else { -- 1.8.1.6