Bug #101595 for Kafka: Kafka::Producer RequiredAcks doesn't take effect

Sat Jan 17 14:40:55 2015 harisekhon [...] gmail.com - Ticket created

Subject:	Kafka::Producer RequiredAcks doesn't take effect
Date:	Sat, 17 Jan 2015 19:40:00 +0000
To:	bug-Kafka [...] rt.cpan.org
From:	Hari Sekhon <harisekhon [...] gmail.com>

Hi, When setting the Kafka::Producer's RequiredAcks to an integer greater than the number of brokers, it seems to have no effect. I would expect this to return an error that there aren't enough brokers similar to how kafka-topics.sh does when attempting to create a topic with higher replication factor than available brokers. At the very least I'd expect a hang waiting for that number of brokers to acknowledge and subsequent timeout error. Is this something that can be set to take effect? Is there a reason why setting a higher integer doesn't cause any problem and the producer just seems to return and allow the program to continue when I would expect it to block? Best Regards, Hari Sekhon http://www.linkedin.com/in/harisekhon

Wed Feb 04 18:38:51 2015 SGLADKOV [...] cpan.org - Correspondence added

Hi, We have prepared a new version 0.8009_1 . Show quoted text

> When setting the Kafka::Producer's RequiredAcks to an integer greater than the number of brokers, it seems to have no effect.

The documentation of "A Guide To The Kafka Protocol" says: "For any number > 1 the server will block waiting for this number of acknowledgements to occur (but the server will never wait for more acknowledgements than there are in-sync replicas)." We rely on this server property. Show quoted text

> I would expect this to return an error that there aren't enough brokers similar to how kafka-topics.sh does when attempting to create a topic with higher replication factor than available brokers.

kafka-topics.sh uses zookeeper information. We do not work with the zookeeper server. Show quoted text

> At the very least I'd expect a hang waiting for that number of brokers to acknowledge and subsequent timeout error. > Is this something that can be set to take effect?

Such a possibility we do not support. New Kafka::Connection->get_metadata and Kafka::Connection->exists_topic_partition methods can be useful for you. Show quoted text

> Is there a reason why setting a higher integer doesn't cause any problem and the producer just seems to return and allow the program to continue when I would expect it to block?

Vide supra. Best Regards, Sergey Gladkov

Wed Feb 04 18:38:51 2015 The RT System itself - Status changed from 'new' to 'open'

Wed Feb 04 18:38:52 2015 SGLADKOV [...] cpan.org - Status changed from 'open' to 'resolved'

Wed Feb 04 18:38:54 2015 SGLADKOV [...] cpan.org - Taken

Wed Feb 04 18:38:55 2015 SGLADKOV [...] cpan.org - Fixed in 0.8009_1 added

Fri Feb 06 19:08:58 2015 harisekhon [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #101595] Kafka::Producer RequiredAcks doesn't take effect
Date:	Sat, 7 Feb 2015 00:08:07 +0000
To:	bug-Kafka [...] rt.cpan.org
From:	Hari Sekhon <harisekhon [...] gmail.com>

Those 2 new methods work well for me in the newer library, I've incorporated them in to my program, thanks! Best Regards, Hari Sekhon http://www.linkedin.com/in/harisekhon On 4 February 2015 at 23:38, Sergey Gladkov via RT <bug-Kafka@rt.cpan.org> wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=101595 > > > Hi, > > We have prepared a new version 0.8009_1 . >

> > When setting the Kafka::Producer's RequiredAcks to an integer greater

> than the number of brokers, it seems to have no effect. > > The documentation of "A Guide To The Kafka Protocol" says: > > "For any number > 1 the server will block waiting for this number of > acknowledgements to occur (but the server will never wait for more > acknowledgements than there are in-sync replicas)." > > We rely on this server property. >

> > I would expect this to return an error that there aren't enough brokers

> similar to how kafka-topics.sh does when attempting to create a topic with > higher replication factor than available brokers. > > kafka-topics.sh uses zookeeper information. We do not work with the > zookeeper server. >

> > At the very least I'd expect a hang waiting for that number of brokers

> to acknowledge and subsequent timeout error.

> > Is this something that can be set to take effect?

> > Such a possibility we do not support. > > New Kafka::Connection->get_metadata and > Kafka::Connection->exists_topic_partition methods can be useful for you. >

> > Is there a reason why setting a higher integer doesn't cause any problem

> and the producer just seems to return and allow the program to continue > when I would expect it to block? > > Vide supra. > > Best Regards, > Sergey Gladkov > >