Bug #78640 for Queue-DBI: Queue lifetime feature && purge feature

Sun Jul 29 06:55:05 2012 bond-spb [...] mail.ru - Ticket created

Subject:	Queue lifetime feature && purge feature
Date:	Sun, 29 Jul 2012 14:55:21 +0400
To:	<bug-queue-dbi [...] rt.cpan.org>
From:	"Sergey Bond" <bond-spb [...] mail.ru>

Hi Guillaume, 1. how about lifetime constraint: my $queue = Queue::DBI->new( 'queue_name' => 'q1', 'database_handle' => $dbh, 'cleanup_timeout' => 600, 'lifetime' => 3600 #sec ); See example http://wiki.qmailtoaster.com/index.php/Queuelifetime 2. it would be nice to have physicaly remove old queue elements: $queue->purge('max_requeue_count' => 10); $queue->purge('lifetime' => 3600); Thanks for your module Regards Sergey

Mon Jul 30 00:04:27 2012 AUBERTG [...] cpan.org - Correspondence added

Hi Sergey, Both are great ideas, I will work on them next weekend but let me know if this is blocking one of your projects and I will see what I can do sooner. Thank you for your feedback and suggestions to improve my module! Guillaume

Mon Jul 30 00:04:28 2012 The RT System itself - Status changed from 'new' to 'open'

Tue Jul 31 12:08:34 2012 bond-spb [...] mail.ru - Correspondence added

Subject:	Re: [rt.cpan.org #78640] Queue lifetime feature && purge feature
Date:	Tue, 31 Jul 2012 20:08:59 +0400
To:	<bug-Queue-DBI [...] rt.cpan.org>
From:	Сергей Бондаренко <bond-spb [...] mail.ru>

Hi Guillaume, Thanks for your answer.There is not necessary to harry, while I can do this operations directly in DB. Sergey Show quoted text

----- Original Message ----- From: "Guillaume B. Aubert via RT" <bug-Queue-DBI@rt.cpan.org> To: <bond-spb@mail.ru> Sent: Monday, July 30, 2012 8:04 AM Subject: [rt.cpan.org #78640] Queue lifetime feature && purge feature

> <URL: https://rt.cpan.org/Ticket/Display.html?id=78640 > > > Hi Sergey, > > Both are great ideas, I will work on them next weekend but let me know > if this is blocking one of your projects and I will see what I can do > sooner. > > Thank you for your feedback and suggestions to improve my module! > > Guillaume >

Sun Aug 05 23:51:22 2012 AUBERTG [...] cpan.org - Correspondence added

Hi Sergey, Thank you for your patience. I just released v1.8.0 of Queue::DBI, which adds 'lifetime' to new() as well as a purge() function. I wrote unit tests for these new features, but would you mind testing this new version and letting me know if you have any problems? I'm very grateful for your suggestions. Guillaume On Tue Jul 31 12:08:34 2012, bond-spb@mail.ru wrote: Show quoted text

> Hi Guillaume, > Thanks for your answer.There is not necessary to harry, while I can do

this Show quoted text

> operations directly in DB. > > Sergey

Mon Aug 06 14:50:57 2012 bond-spb [...] mail.ru - Correspondence added

Subject:	Re: [rt.cpan.org #78640] Queue lifetime feature && purge feature
Date:	Mon, 6 Aug 2012 22:51:29 +0400
To:	<bug-Queue-DBI [...] rt.cpan.org>
From:	Сергей Бондаренко <bond-spb [...] mail.ru>

Hi Guillaume, 1. I am confused a little by a different approach in use of purge parameters: ">" for lifetime and ">=" for max_requeue_count May be better use strict ">" for both, since in retrieve_batch you wrote: line 555: ? 'AND requeue_count <= ' . $dbh->quote( $max_requeue_count ) so no need to purge valid element (10 in your example). 2. that do you think about $queue_element->still_live() function (or a name which is pleasant to you more)? Because while i retrieve batch of elements and do some long processing, lifetime of some last elements can end. Sergey Show quoted text

----- Original Message ----- From: "Guillaume B. Aubert via RT" <bug-Queue-DBI@rt.cpan.org> To: <bond-spb@mail.ru> Sent: Monday, August 06, 2012 7:51 AM Subject: [rt.cpan.org #78640] Queue lifetime feature && purge feature

> <URL: https://rt.cpan.org/Ticket/Display.html?id=78640 > > > Hi Sergey, > > Thank you for your patience. I just released v1.8.0 of Queue::DBI, which > adds 'lifetime' to new() as well as a purge() function. > > I wrote unit tests for these new features, but would you mind testing > this new version and letting me know if you have any problems? I'm very > grateful for your suggestions. > > Guillaume > > On Tue Jul 31 12:08:34 2012, bond-spb@mail.ru wrote:

>> Hi Guillaume, >> Thanks for your answer.There is not necessary to harry, while I can do

> this

>> operations directly in DB. >> >> Sergey

> > >

Wed Aug 08 11:05:49 2012 AUBERTG [...] cpan.org - Correspondence added

Hi Sergey, Show quoted text

> 1. I am confused a little by a different approach in use of purge > parameters: > ">" for lifetime > and ">=" for max_requeue_count > > May be better use strict ">" for both, since in retrieve_batch you wrote: > line 555: ? 'AND requeue_count <= ' . $dbh->quote( $max_requeue_count ) > so no need to purge valid element (10 in your example).

Very good point, I'll make the change to '>' which makes it more consistent and update the documentation/tests. Show quoted text

> 2. that do you think about $queue_element->still_live() function (or a

name Show quoted text

> which is pleasant to you more)? > Because while i retrieve batch of elements and do some long processing, > lifetime of some last elements can end.

That is an excellent idea and should be pretty easy to implement. I will work on both items this weekend and get a new version released for you to take a look at. Thank you so much for your feedback! Guillaume

Sun Aug 12 22:06:18 2012 AUBERTG [...] cpan.org - Correspondence added

Hi Sergey, I just released v1.8.1 of Queue::DBI, which fixes the purge parameters() and implements your suggestion of a function to determine if a queue element is still valid (is_over_lifetime()). Would you mind taking a look and letting me know what you think? I very appreciate your suggestions and input, thank you for helping me make this module better! Guillaume On Mon Aug 06 14:50:57 2012, bond-spb@mail.ru wrote: Show quoted text

> Hi Guillaume, > > 1. I am confused a little by a different approach in use of purge > parameters: > ">" for lifetime > and ">=" for max_requeue_count > > May be better use strict ">" for both, since in retrieve_batch you wrote: > line 555: ? 'AND requeue_count <= ' . $dbh->quote( $max_requeue_count ) > so no need to purge valid element (10 in your example). > > 2. that do you think about $queue_element->still_live() function (or a

name Show quoted text

> which is pleasant to you more)? > Because while i retrieve batch of elements and do some long processing, > lifetime of some last elements can end.

Tue Aug 14 15:44:46 2012 bond-spb [...] mail.ru - Correspondence added

Subject:	Re: [rt.cpan.org #78640] Queue lifetime feature && purge feature
Date:	Tue, 14 Aug 2012 23:45:33 +0400
To:	<bug-Queue-DBI [...] rt.cpan.org>
From:	Сергей Бондаренко <bond-spb [...] mail.ru>

Hi Guillaume, 1. I guess some inconsistency in max_requeue_count() and lifetime() functions. I can: $queue->max_requeue_count( $Queue::DBI::UNLIMITED_RETRIES ); $queue->lifetime( $Queue::DBI::IMMORTAL_LIFE ); but I can't: if ($queue->max_requeue_count() == $Queue::DBI::UNLIMITED_RETRIES) {...}; if ($queue->lifetime() == $Queue::DBI::IMMORTAL_LIFE) {...}; 2. For module completeness I suggest create_queue(), delete_queue() and is_queue_exists() functions, so: - no need to manually insert/delete queue into DB - I can write something like: Queue::DBI::create_queue('my_que') unless Queue::DBI::is_queue_exists('my_que'); - I can additionaly create/delete temporary queues What do you think? Sergey Show quoted text

----- Original Message ----- From: "Guillaume B. Aubert via RT" <bug-Queue-DBI@rt.cpan.org> To: <bond-spb@mail.ru> Sent: Monday, August 13, 2012 6:06 AM Subject: [rt.cpan.org #78640] Queue lifetime feature && purge feature

> <URL: https://rt.cpan.org/Ticket/Display.html?id=78640 > > > Hi Sergey, > > I just released v1.8.1 of Queue::DBI, which fixes the purge parameters() > and implements your suggestion of a function to determine if a queue > element is still valid (is_over_lifetime()). > > Would you mind taking a look and letting me know what you think? I very > appreciate your suggestions and input, thank you for helping me make > this module better! > > Guillaume >

Tue Aug 14 21:11:27 2012 AUBERTG [...] cpan.org - Correspondence added

Hi Sergey, Show quoted text

> 1. I guess some inconsistency in max_requeue_count() and lifetime() > functions. > I can: > $queue->max_requeue_count( $Queue::DBI::UNLIMITED_RETRIES ); > $queue->lifetime( $Queue::DBI::IMMORTAL_LIFE ); > > but I can't: > if ($queue->max_requeue_count() == $Queue::DBI::UNLIMITED_RETRIES) {...}; > if ($queue->lifetime() == $Queue::DBI::IMMORTAL_LIFE) {...};

I noticed it when I was coding the lifetime() method - I actually don't like to have package variables like $Queue::DBI::UNLIMITED_RETRIES exposed, and I wish I had made a different decision when I created the module several years ago. I'm thinking about marking lifetime() and max_requeue_count() as deprecated, and adding separate getters/setters instead: * get_lifetime() * set_lifetime() * get_max_requeue_count() * set_max_requeue_count() That way, you can have: $queue->set_max_requeue_count( undef ); if ( defined( $queue->get_max_requeue_count() ) { ... } What's your opinion? Show quoted text

> 2. For module completeness I suggest create_queue(), delete_queue() and > is_queue_exists() functions, > so: > - no need to manually insert/delete queue into DB > - I can write something like: Queue::DBI::create_queue('my_que') unless > Queue::DBI::is_queue_exists('my_que'); > - I can additionaly create/delete temporary queues

I think this is a great idea - would you mind opening a separate ticket for this? Best, Guillaume

Wed Aug 15 13:06:33 2012 bond-spb [...] mail.ru - Correspondence added

Subject:	Re: [rt.cpan.org #78640] Queue lifetime feature && purge feature
Date:	Wed, 15 Aug 2012 21:07:29 +0400
To:	<bug-Queue-DBI [...] rt.cpan.org>
From:	Сергей Бондаренко <bond-spb [...] mail.ru>

Hi Guillaume, Show quoted text

> I'm thinking about marking lifetime() and max_requeue_count() as > deprecated, and adding separate getters/setters instead:

Certainly, this best idea Show quoted text

> I think this is a great idea - would you mind opening a separate ticket > for this?

Already opened Regards Sergey

Fri Aug 17 21:36:17 2012 AUBERTG [...] cpan.org - Correspondence added

Hi Sergey, Show quoted text

> > I'm thinking about marking lifetime() and max_requeue_count() as > > deprecated, and adding separate getters/setters instead:

> > Certainly, this best idea

I'm going to release version 1.8.2 with the get/set fixes I prepared tonight, and I would like to add a line to thank you in the acknowledgements section of the module. Would this be ok? "Thanks to Sergey Bond for suggesting many features added in version 1.8.x (lifetime constraint, purge() function, get/set functions cleanup)". I'm happy to add your email if you want me to, or use a different spelling for your name. Just let me know! Guillaume

Sat Aug 18 07:36:23 2012 bond-spb [...] mail.ru - Correspondence added

Subject:	Re: [rt.cpan.org #78640] Queue lifetime feature && purge feature
Date:	Sat, 18 Aug 2012 15:37:22 +0400
To:	<bug-Queue-DBI [...] rt.cpan.org>
From:	Сергей Бондаренко <bond-spb [...] mail.ru>

Show quoted text

> "Thanks to Sergey Bond for suggesting many features > added in version 1.8.x (lifetime constraint, purge() function, get/set > functions cleanup)"

This ok. I used earlier Oracle AQ, but for my new project I need in opensoursed (perl, mysql, etc) DB driven QMS. Your module suitable enough. Thank you for acceptance of my suggestions. I hope this is for the benefit of the entire perl community. Sergey

Sat Aug 18 15:18:43 2012 AUBERTG [...] cpan.org - Correspondence added

Hi Sergey, Show quoted text

> > "Thanks to Sergey Bond for suggesting many features > > added in version 1.8.x (lifetime constraint, purge() function, get/set > > functions cleanup)"

> This ok.

I just released version 1.8.2, I will let you take a look and you can let me know if you see anything else. I'm going to try to write the admin module this weekend, I will email you as soon as I have something ready to release. Show quoted text

> I used earlier Oracle AQ, but for my new project I need in opensoursed > (perl, mysql, etc) DB driven QMS. > Your module suitable enough. Thank you for acceptance of my

suggestions. I Show quoted text

> hope this is for the benefit of the entire perl community.

Thank you! It's very gratifying to get good feedback like yours, and your suggestions are definitely helping to improve this module for the community. Have a great weekend, Guillaume

Fri Aug 31 17:51:05 2012 AUBERTG [...] cpan.org - Status changed from 'open' to 'resolved'

Fri Sep 14 10:21:34 2012 bond-spb [...] mail.ru - Correspondence added

Subject:	Re: [rt.cpan.org #78640] Resolved: Queue lifetime feature && purge feature
Date:	Fri, 14 Sep 2012 18:22:15 +0400
To:	<bug-Queue-DBI [...] rt.cpan.org>
From:	Сергей Бондаренко <bond-spb [...] mail.ru>

Hi Guillaume, Seems you forgot required parameter 'created' when create new Queue::DBI::Element object in DBI.pm line 625. Please check. Sergey Show quoted text

----- Original Message ----- From: "Guillaume B. Aubert via RT" <bug-Queue-DBI@rt.cpan.org> To: <bond-spb@mail.ru> Sent: Saturday, September 01, 2012 1:51 AM Subject: [rt.cpan.org #78640] Resolved: Queue lifetime feature && purge feature

> <URL: https://rt.cpan.org/Ticket/Display.html?id=78640 > > > According to our records, your request has been resolved. If you have any > further questions or concerns, please respond to this message. >

Fri Sep 14 10:21:35 2012 The RT System itself - Status changed from 'resolved' to 'open'

Sat Sep 15 20:03:58 2012 AUBERTG [...] cpan.org - Correspondence added

Hi Sergey, Show quoted text

> Seems you forgot required parameter 'created' when create new > Queue::DBI::Element object in DBI.pm line 625.

Excellent catch - I also didn't have tests for the Queue::DBI->cleanup() function. I fixed the code, added tests for the function, and released v2.1.1. Please let me know after you upgrade if you see anything else! Thank you, Guillaume

Sun Sep 16 11:17:17 2012 bond-spb [...] mail.ru - Correspondence added

Subject:	Re: [rt.cpan.org #78640] Queue lifetime feature && purge feature
Date:	Sun, 16 Sep 2012 19:18:03 +0400
To:	<bug-Queue-DBI [...] rt.cpan.org>
From:	Сергей Бондаренко <bond-spb [...] mail.ru>

Hi Guillaume, Show quoted text

> Please let me know after you upgrade if you see anything else!

Please correct croak message in DBI.pm line 675. You just copied it from new method. Sergey

Tue Sep 18 00:16:00 2012 AUBERTG [...] cpan.org - Correspondence added

Hi Sergey, Show quoted text

> Please correct croak message in DBI.pm line 675. You just copied it > from new > method.

Fixed in v2.2.0, thank you! Best, Guillaume

Tue Sep 18 00:16:01 2012 AUBERTG [...] cpan.org - Status changed from 'open' to 'resolved'

Bug #78640 for Queue-DBI: Queue lifetime feature && purge feature

Preferred bug tracker