Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics aren't being sent when using redis as store #101

Open
BRMatt opened this issue Aug 13, 2013 · 0 comments
Open

Metrics aren't being sent when using redis as store #101

BRMatt opened this issue Aug 13, 2013 · 0 comments

Comments

@BRMatt
Copy link
Contributor

BRMatt commented Aug 13, 2013

I upgraded our l2met instance this evening and thought it'd be a good opportunity to setup metchan metrics. As it seems the store related metrics are only available when using redis, I switched our instance from an in-memory store to using a redistogo nano instance. All metchan metrics were then being reported, but l2met stopped sending our normal app metrics:

2013ear-08-14_selection

The point at which the store.throughput crashes is where I disabled redis (I'd stepped away for a while and didn't notice. Luckily we're not using these metrics for alerts! 😄), and the flat line in the outlet.throughput and outlet.latency are where it's not reporting anything, but the averages of averages config for the instrument is filling in the blanks.

During the outage we saw a lot of the following in the logs:

2013-08-13T22:25:20.597416+00:00 app[web.1]: error=redis-time-not-available
2013-08-13T22:25:20.601086+00:00 app[web.1]: error=redis-time-not-available
2013-08-13T22:25:20.615588+00:00 app[web.1]: error=redis-time-not-available
2013-08-13T22:25:20.617556+00:00 app[web.1]: error=redis-time-not-available
2013-08-13T22:25:20.605537+00:00 app[web.1]: error=redis-time-not-available
2013-08-13T22:25:20.612221+00:00 app[web.1]: error=redis-time-not-available
2013-08-13T22:25:20.619599+00:00 app[web.1]: error=redis-time-not-available

Switching back to in memory "fixed" the issue, but it seems odd that this happened.

I'll try running an earlier version of l2met tomorrow, pre adf9daa, but hopefully this information is somewhat useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant