Bug #55177 for Net-IMAP-Simple: unknown return string when using ->put with gmail imap

Wed Mar 03 07:20:07 2010 jquelin [...] cpan.org - Ticket created

Subject:

unknown return string when using ->put with gmail imap

i get the following error when using net::imap::simple with gmail imap: warning unknown return string (id=38): + go ahead and then, the code is doing some searches such as: my ($id) = $imap->search( qq{HEADER Message-ID "<$msgid>"} ); and if the id is not found, then i'm pushing it on the server with: my $res = $imap->put( 'mydir', $msg->string, @flags ); warn $imap->errstr if not $res; the problem is that once the "+ go ahead" message is received from gmail, net::imap::simple looses track of the various exchanges. indeed, the following searches *always* fail, even for some mails that i know are present on gmail (checked with a fresh net::imap::simple object).

Wed Mar 03 08:34:02 2010 jettero [...] cpan.org - Taken

Wed Mar 03 09:16:02 2010 jettero [...] cpan.org - Correspondence added

On Wed Mar 03 07:20:07 2010, JQUELIN wrote: Show quoted text

> the problem is that once the "+ go ahead" message is received from > gmail, net::imap::simple looses track of the various exchanges. indeed, > the following searches *always* fail, even for some mails that i know > are present on gmail (checked with a fresh net::imap::simple object).

I'm not sure I understand this. What is "loses track of the various exchanges?" It sounds like you're saying, when you put a message with $idx and then search for $idx immediately, it's not found? did I get that much right or is there some other problem? -- If riding in an airplane is flying, then riding in a boat is swimming. 114 jumps, 47.2 minutes of freefall, 90.4 freefall miles.

Wed Mar 03 09:16:04 2010 The RT System itself - Status changed from 'new' to 'open'

Wed Mar 03 09:25:39 2010 jquelin [...] cpan.org - Correspondence added

On Wed Mar 03 09:16:02 2010, JETTERO wrote: Show quoted text

> On Wed Mar 03 07:20:07 2010, JQUELIN wrote:

> > the problem is that once the "+ go ahead" message is received from > > gmail, net::imap::simple looses track of the various exchanges. indeed, > > the following searches *always* fail, even for some mails that i know > > are present on gmail (checked with a fresh net::imap::simple object).

> > I'm not sure I understand this. What is "loses track of the various > exchanges?" > > It sounds like you're saying, when you put a message with $idx and then > search for $idx immediately, it's not found? did I get that much right > or is there some other problem?

yes, you're right. sorry for not being clear.

Wed Mar 03 10:03:19 2010 jettero [...] cpan.org - Correspondence added

On Wed Mar 03 09:25:39 2010, JQUELIN wrote: Show quoted text

> yes, you're right. sorry for not being clear.

It's ok. I just wanted to make sure I was working on the right thing. I played with this a bit and I discovered that it's more of a Google bug if it's a bug at all. They only consider message IDs that come in a certain syntactical flavor. I can't comment on the correctness of this, except to say that RFC3501 wants env-message-id to be an nstring, which is (I think) any string or "". my $header = qq(from: jettero\@cpan.org\r\nSubject: test-77\r\nMessage-ID: ); my $footer = qq(\r\n\r\ntest-77\r\n); $goog->put( jet => $header . "test-77" . $footer); $goog->put( jet => $header . "<test-77>" . $footer); $goog->put( jet => $header . "<test-77\@hrm>" . $footer); # finds all three my @id1 = $goog->search(q(SUBJECT "test-77")); # finds only the third one my @id2 = $goog->search(q(HEADER Message-ID "test-77")); So where does that leave this ticket? I'm just going to close it. I think you have to work around Google. There's nothing the client software can do to make this transparent. I'll leave this open for a while in case you have any other thoughts on the topic. -- If riding in an airplane is flying, then riding in a boat is swimming. 114 jumps, 47.2 minutes of freefall, 90.4 freefall miles.

Wed Mar 03 10:23:22 2010 jettero [...] cpan.org - Ticket deleted

Wed Mar 03 10:23:24 2010 jettero [...] cpan.org - Status changed from 'deleted' to 'open'

Wed Mar 03 10:46:36 2010 jquelin [...] cpan.org - Correspondence added

On Wed Mar 03 10:03:19 2010, JETTERO wrote: Show quoted text

> On Wed Mar 03 09:25:39 2010, JQUELIN wrote:

> > yes, you're right. sorry for not being clear.

> > It's ok. I just wanted to make sure I was working on the right thing.

erm. in fact it's more complex than that. i don't find it because the put seems to fail. but the put errstr is: warning unknown return string (id=38): + go ahead so it seems that net::imap::simple doesn't recognize the "+ go ahead" answer that google imap is sending. i'll dig further to see what's going on, and what triggers this.

Wed Mar 03 10:52:36 2010 jettero [...] cpan.org - Correspondence added

On Wed Mar 03 10:46:36 2010, JQUELIN wrote: Show quoted text

> i don't find it because the put seems to fail. but the put errstr is: > > warning unknown return string (id=38): + go ahead

Interesting. Is + go ahead part of the subject or something? I don't understand what that is. Show quoted text

> i'll dig further to see what's going on, and what triggers this.

... use the debug option and possibly attach relevant logs? -- If riding in an airplane is flying, then riding in a boat is swimming. 114 jumps, 47.2 minutes of freefall, 90.4 freefall miles.

Wed Mar 03 11:01:27 2010 jquelin [...] cpan.org - Correspondence added

On Wed Mar 03 10:52:36 2010, JETTERO wrote: Show quoted text

> On Wed Mar 03 10:46:36 2010, JQUELIN wrote:

> > i don't find it because the put seems to fail. but the put errstr is: > > > > warning unknown return string (id=38): + go ahead

> > Interesting. Is + go ahead part of the subject or something? I don't > understand what that is.

no, the + go ahead is not part of anything. grepping for warning in net::imap::simple::ssl show that "+ go ahead" is the result sent by gmail. Show quoted text

> > i'll dig further to see what's going on, and what triggers this.

i found a mail that always makes net:imap:simple fails when put-ting it. i won't attach it here for privacy reasons, but can send it to you if you want. in the meantime, i'm trying to narrow the error. Show quoted text

> ... use the debug option and possibly attach relevant logs?

d'oh. here are the traces: [Local2gmail.pm line 28 in sub new] looking for greeting [Local2gmail.pm line 28 in sub new] got a greeting: * OK Gimap ready for requests from 82.228.201.221 37if288549vws.6\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 908 in sub _send_cmd] 0 LOGIN jquelin "goovoh9I"\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 196 in sub _process_cmd] * CAPABILITY IMAP4rev1 UNSELECT LITERAL+ IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 919 in sub _cmd_ok] * CAPABILITY IMAP4rev1 UNSELECT LITERAL+ IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 196 in sub _process_cmd] 0 OK jquelin@gmail.com authenticated (Success)\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 919 in sub _cmd_ok] 0 OK jquelin@gmail.com authenticated (Success)\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 908 in sub _send_cmd] 1 SELECT "[Gmail]/All Mail"\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 286 in sub _process_cmd] * FLAGS (\Answered \Flagged \Draft \Deleted \Seen)\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 919 in sub _cmd_ok] * FLAGS (\Answered \Flagged \Draft \Deleted \Seen)\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 286 in sub _process_cmd] * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)]\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 919 in sub _cmd_ok] * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)]\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 286 in sub _process_cmd] * OK [UIDVALIDITY 1]\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 919 in sub _cmd_ok] * OK [UIDVALIDITY 1]\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 286 in sub _process_cmd] * 378021 EXISTS\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 919 in sub _cmd_ok] * 378021 EXISTS\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 286 in sub _process_cmd] * 0 RECENT\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 919 in sub _cmd_ok] * 0 RECENT\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 286 in sub _process_cmd] * OK [UIDNEXT 441481]\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 919 in sub _cmd_ok] * OK [UIDNEXT 441481]\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 286 in sub _process_cmd] 1 OK [READ-WRITE] [Gmail]/All Mail selected. (Success)\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 919 in sub _cmd_ok] 1 OK [READ-WRITE] [Gmail]/All Mail selected. (Success)\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 908 in sub _send_cmd] 2 APPEND local2gmail () {73222}\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 527 in sub _process_cmd] + go ahead\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 919 in sub _cmd_ok] + go ahead\r\n [...l/5.10.1/Net/IMAP/Simple.pm line 880 in sub _seterrstr] warning unknown return string (id=2): + go ahead\r\n warning unknown return string (id=2): + go ahead

Wed Mar 03 11:36:04 2010 jettero [...] cpan.org - Correspondence added

On Wed Mar 03 11:01:27 2010, JQUELIN wrote: Show quoted text

> [...l/5.10.1/Net/IMAP/Simple.pm line 880 in sub _seterrstr] warning > unknown return string (id=2): + go ahead\r\n > warning unknown return string (id=2): + go ahead >

Oh, I see it now. Google gives + go ahead and net-imap-server gives + continue. I don't think the "error" matters at all. net-imap-simple::put() returns a 1 when an APPEND works, but it also sets the errstr to "+ go ahead", even when it works. if your put() isn't working, the error message may be spurious (that is, it's showing "+go ahead" instead of what the actual problem is). It is true that net-imap-simple doesn't know what that message is, but it simply moves past it looking for "OK" or "BAD". It's pretty common for net-imap-simple to not understand various lines even after a command success. It ignores a lot of stuff -- If riding in an airplane is flying, then riding in a boat is swimming. 114 jumps, 47.2 minutes of freefall, 90.4 freefall miles.

Thu Mar 04 09:45:59 2010 jettero [...] cpan.org - Correspondence added

On Wed Mar 03 11:36:04 2010, JETTERO wrote: Show quoted text

> It's pretty common for net-imap-simple to not understand various lines > even after a command success. It ignores a lot of stuff

I still think this might be the problem. Even when net-imap-simple succeeds in both putting and finding the message by ID (the third case only); it still gets the warning about + go ahead. $goog->put( jet => $header . "test-77" . $footer); $goog->put( jet => $header . "<test-77>" . $footer); $goog->put( jet => $header . "<test-77\@hrm>" . $footer); -- If riding in an airplane is flying, then riding in a boat is swimming. 114 jumps, 47.2 minutes of freefall, 90.4 freefall miles.

Fri Mar 05 16:17:24 2010 jettero [...] cpan.org - Ticket deleted

Fri Mar 05 16:17:26 2010 jettero [...] cpan.org - Status changed from 'deleted' to 'open'

Fri Mar 05 16:18:24 2010 jettero [...] cpan.org - Correspondence added

I didn't really hear back from you today. I'm going to assume this works (indeed it works for me) and reject the ticket. If this is false, just let me know. -- If riding in an airplane is flying, then riding in a boat is swimming. 114 jumps, 47.2 minutes of freefall, 90.4 freefall miles.

Fri Mar 05 16:18:26 2010 jettero [...] cpan.org - Status changed from 'open' to 'rejected'

Sat Mar 06 03:58:32 2010 jquelin [...] cpan.org - Correspondence added

On Fri Mar 05 16:18:24 2010, JETTERO wrote: Show quoted text

> > I didn't really hear back from you today. I'm going to assume this > works (indeed it works for me) and reject the ticket. > > > If this is false, just let me know.

i haven't had the time to look at it again. i'll try to check this week-end.

Sat Mar 06 03:58:37 2010 The RT System itself - Status changed from 'rejected' to 'open'

Sun Mar 07 12:24:51 2010 jquelin [...] cpan.org - Correspondence added

On Sat Mar 06 03:58:32 2010, JQUELIN wrote: Show quoted text

> i haven't had the time to look at it again. > i'll try to check this week-end.

the problem comes from a given mail which must be mal-formed or whatever: gmail rejects it. i also tried to copy it to gmail using mutt, and mutt closes connection during the transfer, so the mail must be somewhat fishy. you can close, sorry for the noise.

Sun Mar 07 13:34:58 2010 jettero [...] cpan.org - Correspondence added

Show quoted text

> you can close, sorry for the noise. >

No noise at all. I'd really much rather look for a bug than miss one. Thanks for participating!! -- If riding in an airplane is flying, then riding in a boat is swimming. 114 jumps, 47.2 minutes of freefall, 90.4 freefall miles.

Sun Mar 07 13:35:00 2010 jettero [...] cpan.org - Status changed from 'open' to 'rejected'