dropwizard metrics and API timing - dropwizard

I'm trying to do very simple performance measurement of some of my APIs to determine how long they take.
I've added a test resource:
private static final MetricRegistry metrics = new MetricRegistry();
private final Timer responses = metrics.timer("test_responses");
#GET
public void test() {
final Timer.Context context = responses.time();
try {
log.info("sleeping...");
Thread.sleep(10*1000);
} catch (InterruptedException e) {
} finally {
context.stop();//2
}
}
and added the folllowing to my main Application class:
ConsoleReporter reporter = ConsoleReporter.forRegistry(metrics).convertRatesTo(TimeUnit.SECONDS).convertDurationsTo(TimeUnit.MILLISECONDS).build();
reporter.start(10, TimeUnit.SECONDS);
Every ten seconds I see:
6/13/16 11:38:51 AM ============================================================
6/13/16 11:39:01 AM ============================================================
But nothing provided about the "test_responses" metric I created. Any help is greatly appreciated!

Your issue is that you use 2 instances of the metric registry. Look at this example:
public class Application extends io.dropwizard.Application<Configuration>{
#Override
public void run(Configuration configuration, Environment environment) throws Exception {
MetricRegistry metrics = environment.metrics();
environment.jersey().register(new HelloResource(metrics));
ConsoleReporter.forRegistry(metrics).build().start(1, TimeUnit.SECONDS);;
}
public static void main(String[] args) throws Exception {
new Application().run("server", "/home/artur/dev/repo/sandbox/src/main/resources/config/test.yaml");
}
}
I am using the metrics registry that DW creates for you. This MR also includes the Jetty stats.
My resource:
#Path("/test")
#Produces(MediaType.APPLICATION_JSON)
#Consumes(MediaType.APPLICATION_JSON)
public class HelloResource {
private MetricRegistry service;
public HelloResource(MetricRegistry service) {
this.service = service;
}
#GET
public String hello() {
Timer timer = service.timer("test");
try(Context t = timer.time()) {
return "Hello World";
}
}
}
And the output:
test
count = 3
mean rate = 0.89 calls/second
1-minute rate = 0.00 calls/second
5-minute rate = 0.00 calls/second
15-minute rate = 0.00 calls/second
min = 0.00 milliseconds
max = 0.01 milliseconds
mean = 0.00 milliseconds
stddev = 0.00 milliseconds
median = 0.00 milliseconds
75% <= 0.01 milliseconds
95% <= 0.01 milliseconds
98% <= 0.01 milliseconds
99% <= 0.01 milliseconds
99.9% <= 0.01 milliseconds
I have invoked the test method 3 times, and you can see the stats recorded by the MetricRegistry.
I hope that solves your issue.
Regards,
artur

Related

Poor performance of Flux::mergeComparing

I'm migrating my application to Reactor and noticed weird performance issue with Flux::mergeComparing. Here's a benchmark for my existing implementation and Flux alternative.
#Fork(value = 1)
#State(Scope.Thread)
#BenchmarkMode(Mode.Throughput)
#Measurement(iterations = 3)
#OutputTimeUnit(TimeUnit.SECONDS)
public class MergeComparingBenchmark
{
private List<List<Integer>> numbers;
#Setup
public void setUp()
{
numbers = IntStream.range(0, 1_000)
.mapToObj(i -> IntStream.range(0, 10_000))
.map(IntStream::boxed)
.map(stream -> stream.collect(toList()))
.collect(toList());
}
#Benchmark
public List<Integer> myMerge()
{
final List<Stream<Integer>> data = numbers.stream().map(Collection::stream).collect(toList());
final MergingIterator mergingIterator =
new MergingIterator(data);
return Streams.stream(mergingIterator).collect(toList());
}
#Benchmark
public List<Integer> mergeComparing()
{
final Flux<Integer>[] data = numbers.stream().map(Flux::fromIterable)
.toArray(Flux[]::new);
return Flux.mergeComparing(data)
.collectList()
.block();
}
static class MergingIterator implements Iterator<Integer>
{
private final PriorityQueue<Tuple2<Integer, Iterator<Integer>>> queue;
MergingIterator(final List<? extends Stream<Integer>> iterators)
{
this.queue = new PriorityQueue<>(Comparator.comparingInt(entry -> entry.v1));
checkNotNull(iterators).stream()
.map(Stream::iterator)
.filter(Iterator::hasNext)
.forEach(iterator -> queue.add(Tuple.tuple(iterator.next(), iterator)));
}
#Override
public boolean hasNext()
{
return !queue.isEmpty();
}
#Override
public Integer next()
{
final Tuple2<Integer, Iterator<Integer>> element = queue.poll();
final Iterator<Integer> iterator = element.v2;
if (iterator.hasNext()) {
queue.add(Tuple.tuple(iterator.next(), iterator));
}
return element.v1;
}
}
}
And the results are:
MergeComparingBenchmark.mergeComparing thrpt 3 0.181 ± 0.664 ops/s
MergeComparingBenchmark.myMerge thrpt 3 5.146 ± 1.390 ops/s
Why is the reactor case so slow? What can I do to improve it?
I tried to change the subscriber thread pools, changing prefetch size and reactor buffers, but there was not difference.
If your intention is to get a list of sorted integers, then I think you can simply call the collectSortedList method :
return Flux.merge(data)
.collectSortedList()
.block();
I also don't understand why you implemented your own iterator, you basically just need to call the sorted method :
return numbers.stream()
.flatMap(Collection::stream)
.sorted()
.collect(Collectors.toList())
MergeComparing
From the documentation
Merge data from provided Publisher sequences into an ordered merged sequence, by picking the smallest values from each source (as defined by the provided Comparator). This is not a sort(Comparator), as it doesn't consider the whole of each sequences.
Instead, this operator considers only one value from each source and picks the smallest of all these values, then replenishes the slot for that picked source.
It only consider a single element from each sources to do the comparison :
Flux<Integer> flux1 = Flux.just(9, 6, 11);
Flux<Integer> flux2 = Flux.just(10, 2, 13);
Flux.mergeComparing(flux1, flux2)
.doOnNext(System.out::println)
.subscribe();
/**
Output => 9 6 10 2 11 13
Compare 9 and 10 => 9
Compare 6 and 10 => 6
Compare 11 and 10 => 10
Compare 11 and 2 => 2
Compare 11 and 13 => 11
=> 13
**/
This is probably not what you need

Setting a Timer to the minimum timestamp seen

I would like to set a Timer in Event time that fires based on the smallest timestamp seen in the elements within my DoFn.
For performance reasons the Timer API does not support a read() operation, which for the vast majority of use cases is not a required feature. In the small set of use cases where it is needed, for example when you need to set a Timer in EventTime based on the smallest timestamp seen in the elements within a DoFn, we can make use of a State object to keep track of the value.
Java (SDK 2.10.0)
// In this pattern, a Timer is set to fire based on the lowest timestamp seen in the DoFn.
public class SetEventTimeTimerBasedOnEarliestElementTime {
private static final Logger LOG = LoggerFactory
.getLogger(SetEventTimeTimerBasedOnEarliestElementTime.class);
public static void main(String[] args) {
// Create pipeline
PipelineOptions options = PipelineOptionsFactory.
fromArgs(args).withValidation().as(PipelineOptions.class);
// We will start our timer at a fixed point
Instant now = Instant.parse("2000-01-01T00:00:00Z");
// ----- Create some dummy data
// Create 3 elements, incrementing by 1 minute
TimestampedValue<KV<String, Integer>> time_1 = TimestampedValue.of(KV.of("Key_A", 1), now);
TimestampedValue<KV<String, Integer>> time_2 = TimestampedValue
.of(KV.of("Key_A", 2), now.plus(Duration.standardMinutes(1)));
TimestampedValue<KV<String, Integer>> time_3 = TimestampedValue
.of(KV.of("Key_A", 3), now.plus(Duration.standardMinutes(2)));
Pipeline p = Pipeline.create(options);
// Apply a fixed window of duration 10 min and Sum the results
p.apply(Create.timestamped(time_3, time_2, time_1)).apply(
Window.<KV<String, Integer>>into(FixedWindows.<Integer>of(Duration.standardMinutes(10))))
.apply(ParDo.of(new StatefulDoFnThatSetTimerBasedOnSmallestTimeStamp()));
p.run();
}
/**
* Set timer to the lowest value that we see in the stateful DoFn
*/
public static class StatefulDoFnThatSetTimerBasedOnSmallestTimeStamp
extends DoFn<KV<String, Integer>, KV<String, Integer>> {
// Due to performance considerations there is no read on a timer object.
// We make use of this Long value to keep track.
#StateId("currentTimerValue") private final StateSpec<ValueState<Long>> currentTimerValue =
StateSpecs.value(BigEndianLongCoder.of());
#TimerId("timer") private final TimerSpec timer = TimerSpecs.timer(TimeDomain.EVENT_TIME);
#ProcessElement public void process(ProcessContext c,
#StateId("currentTimerValue") ValueState<Long> currentTimerValue,
#TimerId("timer") Timer timer) {
Instant timeStampWeWantToSet = c.timestamp();
//*********** Set Timer
// If the timer has never been set then we set it.
// If the timer has been set but is larger than our current value then we set it.
if (currentTimerValue.read() == null || timeStampWeWantToSet.getMillis() < currentTimerValue
.read()) {
timer.set(timeStampWeWantToSet);
currentTimerValue.write(timeStampWeWantToSet.getMillis());
}
}
#OnTimer("timer") public void onMinTimer(OnTimerContext otc,
#StateId("currentTimerValue") ValueState<Long> currentTimerValue,
#TimerId("timer") Timer timer) {
// Reset the currentTimerValue
currentTimerValue.clear();
LOG.info("Timer # {} fired", otc.timestamp());
}
}
}

Measuring rate of events with Micrometer

In Dropwizard there is something like meter:
https://metrics.dropwizard.io/3.1.0/getting-started/#meters
It lets me measure rate of events just by invoking mark() method on the metric.
How can I do that in Micrometer?
I can use timers, but I don't want to pass Timer.Sample object to wherever place where I need to call stop() method.
The other missing thing in Micrometer comparing to Dropwizard is a metric that can contain a text message, like gauge in Dropwizard.
Micrometer leverages the strengths of modern metrics backends. So the specific answer to your question depends on which you are using. Take Prometheus for example. The backend can calculate the rate for you.
If you are measuring the rate of how often something is happening you can determine that using a Counter. Take the logback_events_total counter as an example. It is merely counting the number for log messages written.
When alerting or graphing you can then write a query like rate(logback_events_total[1m]) and you will be able to see the rate at which logs have been writen at the 1m rate. You have the ability to change to window from 1m, to 5m or 1h without changing the code.
Regarding text based metrics, those aren't useful for alerting (but can be useful when using a join clause). The typical solution in that case is to create a gauge with a value of 1 or 0 and make your text value a tag. For example:
registry.gaugle('app.info', Tags.of("version","1.0.beta3", this, () -> 1.0));
We had the same problem. In DropWizard we were able to use meters to get the rate of events per minute, but in Micrometer we could not find a built-in way that worked for us.
We needed rates for counters and percentiles for timers. The PrometheusMeterRegistry gave us percentiles, but no rates.
So we built our own Gauge that tracks a Counter. Every time getValue() is called, it fetches the value from the counter and adds it to the right bucket with the current timestamp. Then from all available measurements it can compute the rate over the last minute.
It looks like this:
import io.micrometer.core.instrument.Clock;
import io.micrometer.core.instrument.Gauge;
import io.micrometer.core.instrument.MeterRegistry;
import java.util.LinkedList;
import java.util.function.Supplier;
public class OneMinuteRateGauge {
private static final int WINDOW_SECONDS = 60;
private final Supplier<Double> valueSupplier;
private final LinkedList<Bucket> buckets;
private final Clock clock;
public OneMinuteRateGauge(String name, Supplier<Double> valueSupplier, MeterRegistry meterRegistry) {
this(name, valueSupplier, meterRegistry, Clock.SYSTEM);
}
public OneMinuteRateGauge(String name, Supplier<Double> valueSupplier, MeterRegistry meterRegistry, Clock clock) {
this.valueSupplier = valueSupplier;
this.buckets = new LinkedList<>();
Gauge.builder(name, this::getValue).register(meterRegistry);
this.clock = clock;
// Collect one measurement so we have a faster start
getValue();
}
public synchronized double getValue() {
// Update the last bucket or create a new one
long now_millis = clock.monotonicTime() / 1_000_000;
long now_seconds = now_millis / 1_000;
short millis = (short) (now_millis - (now_seconds * 1000));
double value = valueSupplier.get();
if (buckets.size() != 0 && buckets.getLast().getSeconds() == now_seconds) {
buckets.getLast().updateValue(millis, value);
} else {
buckets.addLast(new Bucket(now_seconds, millis, value));
}
// Delete all buckets outside the window except one
while (2 < buckets.size() && buckets.get(1).getSeconds() + WINDOW_SECONDS < now_seconds) {
buckets.pollFirst();
}
if (buckets.size() == 1) {
// Not enough data
return 0;
} else if (now_seconds <= buckets.getFirst().getSeconds() + WINDOW_SECONDS) {
// First bucket is inside the window
return buckets.getLast().getValue() - buckets.getFirst().getValue();
} else {
// Find the weighted average between the first two points
Bucket p0 = buckets.get(0);
Bucket p1 = buckets.get(1);
double px = now_millis - (WINDOW_SECONDS * 1000);
double m = (p1.getValue() - p0.getValue()) / (p1.getTimestampInMillis() - p0.getTimestampInMillis());
double py = m * (px - p0.getTimestampInMillis()) + p0.getValue();
return value - py;
}
}
}
public class Bucket {
private long seconds; // Seconds since 1.1.1970, used as bucket ID
private short millis; // 0-999, used for a more exact calculation
private double value;
public Bucket(long seconds, short millis, double value) {
this.seconds = seconds;
this.millis = millis;
this.value = value;
}
public long getSeconds() {
return seconds;
}
public double getValue() {
return value;
}
public long getTimestampInMillis() {
return seconds * 1000 + millis;
}
public void updateValue(short millis, double value) {
this.millis = millis;
this.value = value;
}
}
An alternative way could have been to use CompositeMeterRegistry on the top level and then add both a PrometheusMeterRegistry and a StepMeterRegistry. Prometheus reports percentiles and Step reports rates. Our monitoring system would then have to query two endpoints.
This was a temporary solution until we modified our monitoring system to read the prometheus endpoint and calculate its own rates.

How do I create user defined counters in Dataflow?

How can I create my own counters in my DoFns?
In my DoFn I'd like to increment a counter every time a condition is met when processing a record. I'd like this counter to sum the values across all records.
You can use Aggregators, and the total values of the counters will show up in the UI.
Here is an example where I experimented with Aggregators in a pipeline that just sleeps numOutputShards workers for sleepSecs seconds. (The GenFakeInput PTransform at the beginning just returns a flattened PCollection<String> of size numOutputShards):
PCollection<String> output = p
.apply(new GenFakeInput(options.getNumOutputShards()))
.apply(ParDo.named("Sleep").of(new DoFn<String, String>() {
private Aggregator<Long> tSleepSecs;
private Aggregator<Integer> tWorkers;
private Aggregator<Long> tExecTime;
private long startTimeMillis;
#Override
public void startBundle(Context c) {
tSleepSecs = c.createAggregator("Total Slept (sec)", new Sum.SumLongFn());
tWorkers = c.createAggregator("Num Workers", new Sum.SumIntegerFn());
tExecTime = c.createAggregator("Total Wallclock (sec)", new Sum.SumLongFn());
startTimeMillis = System.currentTimeMillis();
}
#Override
public void finishBundle(Context c) {
tExecTime.addValue((System.currentTimeMillis() - startTimeMillis)/1000);
}
#Override
public void processElement(ProcessContext c) {
try {
LOG.info("Sleeping for {} seconds.", sleepSecs);
tSleepSecs.addValue(sleepSecs);
tWorkers.addValue(1);
TimeUnit.SECONDS.sleep(sleepSecs);
} catch (InterruptedException e) {
LOG.info("Ignoring caught InterruptedException during sleep.");
}
c.output(c.element());
}}));

Asp.net Mvc OutputCache attribute and sliding expiration

Calling
http://foo/home/cachetest
for
[UrlRoute(Path = "home/cachetest")]
[OutputCache(Duration = 10, VaryByParam = "none")]
public ActionResult CacheTest()
{
return Content(DateTime.Now.ToString());
}
will show the same content for every 10 seconds no matter how often i refresh page.
Is it possible to easily add sliding expiration so it would NOT change after 10 seconds in case i have refreshed the page?
You could create a custom cache filter instead of default OutputCache one. Like this below, note the sliding expiration could be set here. Caveat in that I have not used this for sliding expiration, but works well for other things.
public class CacheFilterAttribute : ActionFilterAttribute
{
private const int Second = 1;
private const int Minute = 60 * Second;
private const int Hour = 60 * Minute;
public const int SecondsInDay = Hour * 24;
/// <summary>
/// Gets or sets the cache duration in seconds.
/// The default is 10 seconds.
/// </summary>
/// <value>The cache duration in seconds.</value>
public int Duration
{
get;
set;
}
public int DurationInDays
{
get { return Duration / SecondsInDay; }
set { Duration = value * SecondsInDay; }
}
public CacheFilterAttribute()
{
Duration = 10;
}
public override void OnActionExecuted(
ActionExecutedContext filterContext)
{
if (Duration <= 0) return;
HttpCachePolicyBase cache =
filterContext.HttpContext.Response.Cache;
TimeSpan cacheDuration = TimeSpan.FromSeconds(Duration);
cache.SetCacheability(HttpCacheability.Public);
cache.SetExpires(DateTime.Now.Add(cacheDuration));
cache.SetMaxAge(cacheDuration);
cache.SetSlidingExpiration(true);
cache.AppendCacheExtension("must-revalidate, proxy-revalidate");
}
}
Been reading the source for the OutputCacheAttribute and I don't think there's an easy way to do this.
You're most likely going to need to create your own solution.
You can't. Internal timer of Cache class spins every 20 secs. I suggest you to try PCache class under PokeIn library. You can set down to 6 secs on it. Also, PCache far more faster in comparison to .NET cache class.

Resources