Skip Menu |

This queue is for tickets about the Net-Amazon-EC2 CPAN distribution.

Report information
The Basics
Id: 107491
Status: patched
Priority: 0/
Queue: Net-Amazon-EC2

People
Owner: Nobody in particular
Requestors: ether [...] cpan.org
sozturk [...] ascllc.net
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: Authentication failure with IAM role
Date: Fri, 2 Oct 2015 11:22:30 -0400
To: bug-Net-Amazon-EC2 [...] rt.cpan.org
From: Selcuk Ozturk <sozturk [...] ascllc.net>
When authenticating from an instance with an IAM role if signature_version=4 is chosen, authentication fails. From the same instance, authentication works if signature_version=2. Authentication with master access key and secret key succeeds with both signature versions. -- -- Selcuk
On Fri Oct 02 11:22:41 2015, sozturk@ascllc.net wrote: Show quoted text
> When authenticating from an instance with an IAM role if > signature_version=4 is chosen, authentication fails. From the same > instance, authentication works if signature_version=2. > > Authentication with master access key and secret key succeeds with both > signature versions.
Thanks for the report - would it be possible to submit a failing test case? I don't ever use IAM authentication so its especially difficult for me to design a test case. Thanks.
Subject: Re: [rt.cpan.org #107491] Authentication failure with IAM role
Date: Fri, 2 Oct 2015 14:56:37 -0400
To: bug-Net-Amazon-EC2 [...] rt.cpan.org
From: Selcuk Ozturk <sozturk [...] ascllc.net>
If you can explain what you mean by a test case, I might be able to do it. -- Selcuk On 10/2/15 2:46 PM, Mark Allen via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=107491 > > > On Fri Oct 02 11:22:41 2015, sozturk@ascllc.net wrote:
>> When authenticating from an instance with an IAM role if >> signature_version=4 is chosen, authentication fails. From the same >> instance, authentication works if signature_version=2. >> >> Authentication with master access key and secret key succeeds with both >> signature versions.
> Thanks for the report - would it be possible to submit a failing test case? I don't ever use IAM authentication so its especially difficult for me to design a test case. > > Thanks. > > >
On Fri Oct 02 14:56:50 2015, sozturk@ascllc.net wrote: Show quoted text
> If you can explain what you mean by a test case, I might be able to do > it.
I mean a .t file which has a failing test case for the authentication you discussed - take a look at https://github.com/mrallen1/net-amazon-ec2/blob/master/t/04_live_v4.t for inspiration of what this might look like - the test file would be a a series of operation that are repeatable so that when I work on this bug I can test a solution until the test case(s) pass the test. Thanks for any help you can provide. Mark
Subject: Re: [rt.cpan.org #107491] Authentication failure with IAM role
Date: Fri, 2 Oct 2015 15:52:33 -0400
To: bug-Net-Amazon-EC2 [...] rt.cpan.org
From: Selcuk Ozturk <sozturk [...] ascllc.net>
I have pasted a very simple test case at the end of this message. Of course, the point is, this test needs to be run on an instance that has been launched with an IAM role (as explained here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html), so Net::Amazon::EC2 can obtain the credentials automatically. Also, I think the region needs to be the same region where the instance is running. I think these role based instance credentials are limited to the region they are in. But, I may be wrong on that. When I run the below test on my instance launched with an IAM role allowing all describe type access in us-east-1a, it fails with authentication failure. However, if I just change the 'signature_version = 2', it passes. -- Selcuk On 10/2/15 2:59 PM, Mark Allen via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=107491 > > > On Fri Oct 02 14:56:50 2015, sozturk@ascllc.net wrote:
>> If you can explain what you mean by a test case, I might be able to do >> it.
> I mean a .t file which has a failing test case for the authentication you discussed - take a look at > https://github.com/mrallen1/net-amazon-ec2/blob/master/t/04_live_v4.t > > for inspiration of what this might look like - the test file would be a a series of operation > that are repeatable so that when I work on this bug I can test a solution until the test > case(s) pass the test. > > Thanks for any help you can provide. > > Mark
------- cut use strict; use blib; use Test::More; BEGIN { plan tests => 3; use_ok( 'Net::Amazon::EC2' ); }; my $ec2 = eval { Net::Amazon::EC2->new( region => 'us-east-1', ssl => 1, #debug => 1, signature_version => 4, return_errors => 1, ); }; isa_ok($ec2, 'Net::Amazon::EC2'); my $regions = $ec2->describe_regions(); my $seen_region = 0; if (ref($regions) eq 'Net::Amazon::EC2::Errors') { foreach my $err (@{$regions->errors}) { print $err->message . "\n"; } fail("Describing regions"); } else { foreach my $region (@{$regions}) { if ($region->region_name eq 'us-east-1') { $seen_region = 1; } } ok($seen_region == 1, "Describing regions"); } ------ cut
On Fri Oct 02 15:52:47 2015, sozturk@ascllc.net wrote: Show quoted text
> I have pasted a very simple test case at the end of this message. Of > course, the point is, this test needs to be run on an instance that > has > been launched with an IAM role (as explained here: > http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for- > amazon-ec2.html), > so Net::Amazon::EC2 can obtain the credentials automatically. > > Also, I think the region needs to be the same region where the > instance > is running. I think these role based instance credentials are limited > to > the region they are in. But, I may be wrong on that. > > When I run the below test on my instance launched with an IAM role > allowing all describe type access in us-east-1a, it fails with > authentication failure. However, if I just change the > 'signature_version > = 2', it passes.
Thanks! That's super helpful.
This bug affects of multiple users of `ec2-consistent-snapshot` who would like to use IAM roles in newer regions where only Signature Version 4 is supported https://github.com/alestic/ec2-consistent-snapshot/issues/76 I'll take a look a the Perl for Net::Amazon::EC2 now to see if anything stands out.
I discovered part of the bug, but there is still more to debug. First, the definite bug I found: "temp_creds" is defined with "lazy => 1", so that the values will be looked up when they needed. But in _sign_v4, because calling temp_creds, there's a call to "has_temp_creds" to check if the temp creds *already exist*. Since they don't, temp_creds is never called. Thus, you can see the resulting request does not contain "X-Amz-Security-Token" when it should. I tested this was the issue by removing "lazy => 1", which made X-Amz-Security-Token" start appearing in the resulting request as desired. But the requests still fail anyway. I started to look further, but I'll stop and ask, why not use AWS::Signature4 for generating the signature? It appears to have solved Signature4/IAM role issue. All this module would have to do is pass it the secret token.
On Fri Dec 30 14:55:39 2016, MARKSTOS wrote: Show quoted text
> I discovered part of the bug, but there is still more to debug. First, > the definite bug I found: > > "temp_creds" is defined with "lazy => 1", so that the values will be > looked up when they needed. > > But in _sign_v4, because calling temp_creds, there's a call to > "has_temp_creds" to check if the temp creds *already exist*. Since > they don't, temp_creds is never called. Thus, you can see the > resulting request does not contain "X-Amz-Security-Token" when it > should. > > I tested this was the issue by removing "lazy => 1", which made X-Amz- > Security-Token" start appearing in the resulting request as desired. > > But the requests still fail anyway. > > I started to look further, but I'll stop and ask, why not use > AWS::Signature4 for generating the signature? It appears to have > solved Signature4/IAM role issue. All this module would have to do is > pass it the secret token.
Thanks for looking at this issue. Sorry - I obviously had forgotten about it! Also, thank you for debugging things this far. I can't remember why but IIRC when I was working on V4 support either the modules that were currently available on CPAN were not compatible with Moose for some reason or were poorly written or both. I have long thought outsourcing the signature work to their own modules would be a nice change but never got around to encapsulating them and making it work. I would definitely welcome a PR implementing that change though, if you feel like working on it. If not, I will try to get it done in the next few weeks. Thanks again! Mark
On Fri Dec 30 14:55:39 2016, MARKSTOS wrote: I have a PR up that I need help testing with https://github.com/mrallen1/net-amazon-ec2/pull/60 Since I don't use IAM roles, it's a bit difficult for me to directly test this code. Any testing help on that front would be great. Thanks.
On Thu Dec 28 14:33:32 2017, mallen wrote: Show quoted text
> On Fri Dec 30 14:55:39 2016, MARKSTOS wrote: > > I have a PR up that I need help testing with > > https://github.com/mrallen1/net-amazon-ec2/pull/60 > > Since I don't use IAM roles, it's a bit difficult for me to directly > test this code. > > Any testing help on that front would be great. > > Thanks.
OK, I did get a tester and I released 0.35 to CPAN today which I believe fixes this issue. Please give it a try. Thanks.