› owit logs level=error service=api --last 1h --limit 25
loki14:32:01errorapi-gwa3f9e12bupstream timeout: payment-svc (3042ms)
datadog14:32:01errorpaymenta3f9e12bstripe: connection reset by peer
appinsights14:32:03errorapi-gwa3f9e12bretry payment-svc: attempt 2/3
cloudwatch14:32:05errorpaymenta3f9e12bstripe webhook delivery failed: 503
loki14:33:11errorauthb7c1d24ejwt: signing key not found in JWKS
datadog14:33:13errorauthb7c1d24eoidc provider returned 502 Bad Gateway
appinsights14:33:14errorapi-gwb7c1d24e401 upstream: auth-svc unavailable
cloudwatch14:33:15errorauthb7c1d24edependency health check failed
loki14:38:44errorinventoryc0a5f33ddeadlock detected on orders table
datadog14:38:45errorinventoryc0a5f33dpostgres: deadlock, transaction aborted
appinsights14:38:46errorapi-gwc0a5f33d500 inventory-svc: internal server error
cloudwatch14:38:47errorinventoryc0a5f33dtx rolled back after 3 retries
loki14:42:09errorcheckoutd1b8e71finventory hold expired: order aborted
datadog14:42:10errorcheckoutd1b8e71fsaga: compensating transaction started
appinsights14:42:11errorcheckoutd1b8e71fsaga rollback: payment already captured
cloudwatch14:42:13errorcheckoutd1b8e71forder.failed enqueued to DLQ
loki14:51:22errornotifiere2c9a80aSMTP rejected: 550 address rejected
datadog14:51:23errornotifiere2c9a80asendgrid: daily rate limit exceeded
appinsights14:51:24errorapi-gwe2c9a80anotification skipped: quota hit
cloudwatch14:51:25errornotifiere2c9a80aSES throttling: message dropped
loki14:57:38errorsearchf3d0b91celasticsearch cluster status: red
datadog14:57:39errorsearchf3d0b91cquery timeout after 5000ms
appinsights14:57:41errorapi-gwf3d0b91c503 search-svc: upstream unavailable
cloudwatch14:57:42errorsearchf3d0b91ccircuit breaker open: search-svc
loki14:58:55errorapi-gwg4e1c02drate limit: 429 sent to 10.0.2.15 (burst)
⚠ 4 backends queried · 25 events · 0 backend errors
›
› owit metrics service=api --last 30m --step 2m
prometheus14:30:00http_request_duration_p99142msservice=api env=prod
prometheus14:30:00http_request_duration_p5018msservice=api env=prod
prometheus14:30:00http_error_rate0.04%service=api env=prod
datadog14:30:00http_request_duration_p99138msservice:api env:prod
datadog14:30:00http_error_rate0.03%service:api env:prod
cloudwatch14:30:00ALBTargetResponseTime144msTargetGroup=api-prod
cloudwatch14:30:00HTTPCode_Target_5XX_Count3TargetGroup=api-prod
azmonitor14:30:00requests/duration141msroleName=api
azmonitor14:30:00requests/failed2roleName=api
prometheus14:32:00http_request_duration_p99289ms ↑service=api env=prod
datadog14:32:00http_request_duration_p99301ms ↑service:api env:prod
cloudwatch14:32:00ALBTargetResponseTime310ms ↑TargetGroup=api-prod
azmonitor14:32:00requests/duration294ms ↑roleName=api
prometheus14:34:00http_error_rate1.2% ↑service=api env=prod
datadog14:34:00http_error_rate1.4% ↑service:api env:prod
cloudwatch14:34:00HTTPCode_Target_5XX_Count18 ↑TargetGroup=api-prod
azmonitor14:36:00requests/failed22 ↑roleName=api
prometheus14:38:00http_request_duration_p993042ms ↑service=api env=prod
datadog14:38:00http_request_duration_p993108ms ↑service:api env:prod
cloudwatch14:38:00ALBTargetResponseTime3089ms ↑TargetGroup=api-prod
azmonitor14:38:00requests/duration3021ms ↑roleName=api
prometheus14:40:00http_error_rate8.9% ↑service=api env=prod
⚠ 4 backends queried · 22 data points · 0 backend errors
›
› owit traces service=checkout status=error --last 1h --limit 20
tempo14:32:01a3f9e12b3042mserror12checkout → payment-svc → stripe-proxy
tempo14:32:01a3f9e12b841mserror5payment-svc → stripe-proxy
tempo14:32:01a3f9e12b3041mserror3stripe-proxy [connect timeout]
jaeger14:32:01a3f9e12b3044mserror11checkout.processPayment
jaeger14:32:01a3f9e12b843mserror4payment-svc.chargeCard
datadog14:32:01a3f9e12b3048mserror—checkout [apm]
tempo14:33:14b7c1d24e201mserror6api-gw → auth-svc → oidc-provider
tempo14:33:14b7c1d24e199mserror3auth-svc → oidc-provider [502]
jaeger14:33:14b7c1d24e203mserror5api-gw.authenticate
datadog14:33:14b7c1d24e198mserror—auth-svc [apm]
tempo14:38:44c0a5f33d1204mserror8inventory-svc → postgres [deadlock]
jaeger14:38:44c0a5f33d1201mserror7inventory-svc.createHold
datadog14:38:45c0a5f33d1208mserror—inventory-svc [apm]
zipkin14:38:44c0a5f33d1210mserror9inventory-svc [deadlock detected]
tempo14:42:09d1b8e71f2103mserror15checkout → inventory → payment-svc
tempo14:42:09d1b8e71f412mserror4inventory-svc [hold expired]
jaeger14:42:09d1b8e71f2109mserror14checkout.placeOrder [saga rollback]
datadog14:42:10d1b8e71f2115mserror—checkout [apm]
zipkin14:42:09d1b8e71f2098mserror13checkout → saga rollback → payment-svc
tempo14:57:38f3d0b91c500mserror6api-gw → search-svc [circuit open]
⚠ 4 backends queried · 20 spans · zipkin: b7c1d24e not found (auth-svc not instrumented)
›