Bug #94294 for Net-Statsd: Multi-metric gauge feature

Sat Mar 29 01:50:14 2014 vm [...] iki.fi - Ticket created

Subject:	Multi-metric gauge feature
Date:	Sat, 29 Mar 2014 07:49:52 +0200
To:	bug-Net-Statsd [...] rt.cpan.org
From:	Ville Mattila <vm [...] iki.fi>

Hi, Net::Statsd v0.08 works great but it's currently missing support for the Multi-Metric Packets feature of statsd, described at https://github.com/etsy/statsd/blob/master/docs/metric_types.md#multi-metric-packets A simple(?) patch which makes Net::Statsd::gauge() accept multiple metrics and modifies send() and _sample_data() accordingly to pack all of metrics into single UDP packet is attached. AFAIK the module API would remain backwards compatible. (The code works for me, but I've never used the sampling features or anything but gauge() so I'm not at all sure if the changes are correct/sane.) Rationale: The possibility to pack multiple metrics into single UDP packet is essential for me, as I'm collecting negative temperature readings from remote locations (in Finland the outdoor temperature ⁰C during winters typically is negative) which have very unreliable network connections; statsd interprets gauge values with '+' or '-' sign as increment/decrement and therefore to set a gauge to a negative value it must be set to zero first. Now it is not a problem if I lose some readings, but if the zero and the actual negative value are sent in separate UDP packets my temperature data will become badly skewed if either one of the UDP packets is lost or reordered in the network transit. NB: The suggested patch is rather ugly, especially in how the $value's passed to send() and _sample_data() can be either scalars (as in 0.08) or array refs. Cleaner implementation wrt the module internals might be good to have. Thanks, -- Ville Mattila

Download 0x81DDC625.asc
application/pgp-keys 4.5k

Message body not shown because it is not plain text.

Message body is not shown because sender requested not to inline it.

Download signature.asc
application/pgp-signature 836b

Message body not shown because it is not plain text.

Wed Jun 04 11:30:47 2014 cosimo [...] cpan.org - Correspondence added

Thank you for this contribution! I admit I'm a bit afraid to break Net::Statsd, but I'm going to have a look at this in more detail :-)

Wed Jun 04 11:30:47 2014 The RT System itself - Status changed from 'new' to 'open'

Thu Jun 05 10:04:57 2014 cosimo [...] cpan.org - Correspondence added

Hi again! I looked at the code and tried to build a suitable test case to demonstrate that: 1) nothing breaks 2) the multi-metric functionality works I found out that (please correct me if I'm wrong): 1) if you send multiple values for the *same* gauge (core.temp=0, core.temp=56), then the two values will be sent in a single packet 2) if you send multiple metrics that have different name (core.temp, freezer.temp) then the code doesn't aggregate them in a single packet. Did I understand correctly? And, is that what you intended?

Thu Jun 05 10:12:23 2014 cosimo [...] cpan.org - Correspondence added

Here is a pointer to the latest code, tagged "multi-metric": https://github.com/cosimo/perl5-net-statsd/tree/multi-metric in case you want to try it out and run the new tests.

Mon Jun 16 23:42:05 2014 vm [...] iki.fi - Correspondence added

Subject:	Re: [rt.cpan.org #94294] Multi-metric gauge feature
Date:	Tue, 17 Jun 2014 06:41:43 +0300
To:	bug-Net-Statsd [...] rt.cpan.org
From:	Ville Mattila <vm [...] iki.fi>

Hi, On 2014-06-05 17:04, Cosimo Streppone via RT wrote: Show quoted text

> I looked at the code and tried to build a suitable test case to demonstrate that: > 1) nothing breaks > 2) the multi-metric functionality works > > I found out that (please correct me if I'm wrong): > > 1) if you send multiple values for the *same* gauge (core.temp=0, core.temp=56), then the two values will be sent in a single packet > 2) if you send multiple metrics that have different name (core.temp, freezer.temp) then the code doesn't aggregate them in a single packet. > > Did I understand correctly?

Yes, that's correct. Show quoted text

> And, is that what you intended?

Yes. It is necessary to be able to send two values for the *same* gauge in a single packet. An example to justify this: If you just send outdoor.temp=-10 to statsd it will interpret as difference to previous value of the gauge because the value has a sign (see https://github.com/etsy/statsd/blob/master/docs/metric_types.md#gauges). Therefore you must send outdoor.temp=0 just before you send the actual negative value to it. If the values (outdoor.temp=0, outdoor.temp=-10) are sent in separate UDP packets losing either one of them (because of network connectivity problems) will break your data. By packing the zero and the actual negative value both in a single packet protects your data from becoming skewed because you always either receive or lose both of the values. Thanks, -- Ville Mattila

Download signature.asc
application/pgp-signature 819b

Message body not shown because it is not plain text.

Thu Apr 23 10:43:36 2015 cosimo [...] cpan.org - Correspondence added

Version 0.11, soon on CPAN, has support for multi-value gauges.

Thu Apr 23 10:43:37 2015 cosimo [...] cpan.org - Status changed from 'open' to 'resolved'