Alerts
An alert is a row before it is an email. It is written in the same transaction as the event that caused it, so nothing about a failed delivery, an unreachable mail server, or an instance going to sleep mid-send can lose it.
Latency, honestly
Detection runs when an event arrives, because that is the only moment this service is reliably running — it sleeps after fifteen idle minutes. So an alert is not raised the instant a spike begins; it is raised on the next event that reaches a woken instance, and the wake itself takes about a hundred seconds. A spike that starts and stops entirely within a quiet period is never seen at all.
- 10d agospikecheckout-api
IllegalStateException: could not reserve stock for cart <uuid>
ewma · 5 this hour against a baseline of 0.29 · score 10.00
emailed · 1 attempt
- 10d agowent quietpayments-api
DataAccessResourceFailureException: connection pool exhausted acquiring connection for payment <uuid>
recorded only, mail was not configured at the time
- 10d agowent quietnotification-worker
NullPointerException: Cannot invoke "String.length()" because "template" is null for user <uuid>
recorded only, mail was not configured at the time
- 10d agowent quietmedia-transcoder
RangeError: Invalid array length while decoding segment <num>
recorded only, mail was not configured at the time
- 10d agowent quietcheckout-api
IllegalStateException: could not reserve stock for cart <uuid>
recorded only, mail was not configured at the time
- 11d agospikemedia-transcoder
RangeError: Invalid array length while decoding segment <num>
poisson · 25 this hour against a baseline of 12.29 · score 3.02
recorded only, mail was not configured at the time
- 11d agospikesearch-indexer
TypeError: Cannot read properties of undefined (reading 'documentId') for job <num>
poisson · 19 this hour against a baseline of 8.25 · score 3.03
recorded only, mail was not configured at the time
- 11d agospikepayments-api
DataAccessResourceFailureException: connection pool exhausted acquiring connection for payment <uuid>
poisson · 16 this hour against a baseline of 6.38 · score 3.02
recorded only, mail was not configured at the time
- 11d agospikenotification-worker
NullPointerException: Cannot invoke "String.length()" because "template" is null for user <uuid>
poisson · 40 this hour against a baseline of 22.83 · score 3.15
recorded only, mail was not configured at the time
- 11d agospikenotification-worker
NullPointerException: Cannot invoke "String.length()" because "template" is null for user <uuid>
poisson · 38 this hour against a baseline of 21.27 · score 3.17
recorded only, mail was not configured at the time
- 11d agospikemedia-transcoder
RangeError: Invalid array length while decoding segment <num>
poisson · 24 this hour against a baseline of 11.14 · score 3.26
recorded only, mail was not configured at the time
- 11d agospikesearch-indexer
TypeError: Cannot read properties of undefined (reading 'documentId') for job <num>
poisson · 19 this hour against a baseline of 7.90 · score 3.25
recorded only, mail was not configured at the time
- 11d agospikenotification-worker
NullPointerException: Cannot invoke "String.length()" because "template" is null for user <uuid>
poisson · 37 this hour against a baseline of 20.38 · score 3.23
recorded only, mail was not configured at the time
- 11d agospikenotification-worker
NullPointerException: Cannot invoke "String.length()" because "template" is null for user <uuid>
poisson · 34 this hour against a baseline of 18.32 · score 3.18
recorded only, mail was not configured at the time
- 11d agospikenotification-worker
NullPointerException: Cannot invoke "String.length()" because "template" is null for user <uuid>
poisson · 32 this hour against a baseline of 17.11 · score 3.08
recorded only, mail was not configured at the time
- 11d agospikesearch-indexer
TypeError: Cannot read properties of undefined (reading 'documentId') for job <num>
poisson · 18 this hour against a baseline of 7.31 · score 3.23
recorded only, mail was not configured at the time
- 11d agospikecheckout-api
IllegalStateException: could not reserve stock for cart <uuid>
poisson · 19 this hour against a baseline of 7.88 · score 3.27
recorded only, mail was not configured at the time
- 11d agospikenotification-worker
NullPointerException: Cannot invoke "String.length()" because "template" is null for user <uuid>
poisson · 26 this hour against a baseline of 12.38 · score 3.32
recorded only, mail was not configured at the time
- 11d agospikemedia-transcoder
RangeError: Invalid array length while decoding segment <num>
poisson · 18 this hour against a baseline of 7.06 · score 3.40
recorded only, mail was not configured at the time
- 11d agospikenotification-worker
NullPointerException: Cannot invoke "String.length()" because "template" is null for user <uuid>
poisson · 25 this hour against a baseline of 11.86 · score 3.24
recorded only, mail was not configured at the time
- 11d agospikesearch-indexer
TypeError: Cannot read properties of undefined (reading 'documentId') for job <num>
poisson · 17 this hour against a baseline of 6.50 · score 3.37
recorded only, mail was not configured at the time
- 11d agospikepayments-api
DataAccessResourceFailureException: connection pool exhausted acquiring connection for payment <uuid>
poisson · 13 this hour against a baseline of 4.08 · score 3.49
recorded only, mail was not configured at the time
- 11d agospikenotification-worker
NullPointerException: Cannot invoke "String.length()" because "template" is null for user <uuid>
poisson · 23 this hour against a baseline of 10.38 · score 3.31
recorded only, mail was not configured at the time
- 11d agowent quietsession-store
Error: session <uuid> evicted before write completed
recorded only, mail was not configured at the time
- 11d agospikenotification-worker
NullPointerException: Cannot invoke "String.length()" because "template" is null for user <uuid>
poisson · 22 this hour against a baseline of 9.82 · score 3.26
recorded only, mail was not configured at the time
- 12d agonew errorsession-store
Error: session <uuid> evicted before write completed
recorded only, mail was not configured at the time
- 12d agonew errormedia-transcoder
RangeError: Invalid array length while decoding segment <num>
recorded only, mail was not configured at the time
- 12d agonew errorsearch-indexer
TypeError: Cannot read properties of undefined (reading 'documentId') for job <num>
recorded only, mail was not configured at the time
- 12d agonew errorpayments-api
DataAccessResourceFailureException: connection pool exhausted acquiring connection for payment <uuid>
recorded only, mail was not configured at the time
- 12d agonew errornotification-worker
NullPointerException: Cannot invoke "String.length()" because "template" is null for user <uuid>
recorded only, mail was not configured at the time
- 12d agonew errorcheckout-api
IllegalStateException: could not reserve stock for cart <uuid>
recorded only, mail was not configured at the time
- 12d agonew errorsession-store
Error: session <uuid> evicted before write completed
recorded only, mail was not configured at the time
- 12d agonew errormedia-transcoder
RangeError: Invalid array length while decoding segment <num>
recorded only, mail was not configured at the time
- 12d agonew errorcheckout-api
IllegalStateException: could not reserve stock for cart <uuid>
recorded only, mail was not configured at the time
- 14d agospikebilling-worker
<unknown>: Queue lag above threshold: <num> messages on invoices at <ip>
poisson · 5 this hour against a baseline of 0.10 · score 7.12
emailed · 1 attempt
- 14d agospikeweb-ui
TypeError: Cannot read properties of undefined (reading 'id') at <timestamp>
poisson · 5 this hour against a baseline of 0.10 · score 7.12
emailed · 1 attempt
- 14d agospikecheckout-api
NullPointerException: Cannot invoke "String.length()" because "promoCode" is null for cart <uuid>
poisson · 5 this hour against a baseline of 0.10 · score 7.12
recorded only, mail was not configured at the time