Climate metrics jobsTimes in UTC
Queued
5,775
oldest waiting 58 min
Running
49
on 10 of 10 worker pods
Longest wait now
58 min
oldest queued job
Succeeded, last hour
2,845
jobs finished
Failure rate, 24 h
8.5 %
1,351 failed of 15.8K
Run time p95, 24 h
9.6 s
median 4.7 s

1 Oct 23:00 – now · 14.5K succeeded, 1,351 failed, run time p95 9.6 s, longest wait 1 h 39 min

Queue

peak per 1 h

  • Queued
  • Running
Table
TimeQueuedRunning
2 Oct 21:009,99066
2 Oct 22:008,29258

Worker pods

peak per 1 h

  • Ready
  • Busy
  • Starting
  • Waiting for a node
Table
TimeReadyBusyStartingWaiting for a node
2 Oct 21:00101070
2 Oct 22:00101000

Throughput

jobs per 1 h

  • Succeeded
  • Failed
  • Cancelled
  • Submitted
Table
TimeSucceededFailedCancelledSubmitted
2 Oct 06:005840010.1K
2 Oct 07:006,036000
2 Oct 08:003,4681200
2 Oct 13:008008
2 Oct 14:00991901,000
2 Oct 15:0010200102
2 Oct 17:0030000300
2 Oct 18:00500050
2 Oct 19:001001
2 Oct 20:0010800108
2 Oct 21:001,63415010K
2 Oct 22:001,2091,31500

Run time

succeeded jobs, started to finished

  • Median
  • 95th percentile
  • Longest
Table
TimeMedian95th percentileLongest
2 Oct 06:004.9 s9.6 s41 s
2 Oct 07:004.5 s9.6 s56 s
2 Oct 08:004.1 s9.4 s42 s
2 Oct 13:003.5 s4.8 s4.8 s
2 Oct 14:005.9 s9.7 s47 s
2 Oct 15:006.5 s10.0 s56 s
2 Oct 17:006.1 s9.8 s55 s
2 Oct 18:004.3 s9.8 s43 s
2 Oct 19:005.8 s5.8 s5.8 s
2 Oct 20:006.1 s9.8 s56 s
2 Oct 21:006.4 s9.8 s59 s
2 Oct 22:005.6 s9.8 s1 min 1 s

Wait time

created to started

  • 95th percentile (started)
  • Longest (started)
  • Oldest still queued
Table
Time95th percentile (started)Longest (started)Oldest still queued
2 Oct 06:004 min 42 s4 min 57 s–
2 Oct 07:0060 min1 h 2 min–
2 Oct 08:001 h 37 min1 h 39 min–
2 Oct 13:001.9 s2.0 s–
2 Oct 14:0019 min19 min–
2 Oct 15:004 min 33 s4 min 39 s–
2 Oct 17:009 min 40 s10 min–
2 Oct 18:002 min 40 s2 min 40 s–
2 Oct 19:000.4 s0.4 s–
2 Oct 20:004 min 56 s6 min 6 s–
2 Oct 21:0027 min27 min26 min
2 Oct 22:0058 min58 min58 min

Failures

by kind, per 1 h

Details →
  • compute_error
  • invalid_asset
  • worker_lost
  • internal
Table
Timecompute_errorinvalid_assetworker_lostinternal
2 Oct 08:0012000
2 Oct 14:009000
2 Oct 21:0015000
2 Oct 22:001,315000

Next in the queue

first in, first out

AssetSubmittedWaitingAttempts
USA_000862 Oct 21:33:1758 min0
FRA_000302 Oct 21:33:1758 min0
BEL_000022 Oct 21:33:1758 min0
BRA_000032 Oct 21:33:1758 min0
CPV_000012 Oct 21:33:1758 min0
GUF_000012 Oct 21:33:1758 min0
PRK_000012 Oct 21:33:1758 min0
TUR_000012 Oct 21:33:1758 min0
USA_000142 Oct 21:33:1758 min0
BRA_000012 Oct 21:33:1758 min0

and 5,765 more

Worker pods now

10 ready, 0 starting, 0 waiting for a node

PodStateJobAge
climate-metrics-worker-standby-57c4499f86-g7vrh
standby · ip-10-40-15-254
readyUSA_000052 h 37 min
climate-metrics-worker-7b6cddc9fc-2wn4r
pool · ip-10-40-20-253
readyGBR_0002757 min
climate-metrics-worker-7b6cddc9fc-6sbnv
pool · ip-10-40-20-253
readyFRA_0002057 min
climate-metrics-worker-7b6cddc9fc-7rvrq
pool · ip-10-40-20-253
readyPSE_0000157 min
climate-metrics-worker-7b6cddc9fc-8fsxj
pool · ip-10-40-20-253
readyPSE_0000157 min
climate-metrics-worker-7b6cddc9fc-fd7hr
pool · ip-10-40-20-253
readyFRA_0002957 min
climate-metrics-worker-7b6cddc9fc-v2mxp
pool · ip-10-40-20-253
readyUSA_0001757 min
climate-metrics-worker-7b6cddc9fc-w7kzj
pool · ip-10-40-20-253
readyITA_0000557 min
climate-metrics-worker-7b6cddc9fc-x5zvn
pool · ip-10-40-93-92
readySGP_0002854 min
climate-metrics-worker-7b6cddc9fc-zxdzw
pool · ip-10-40-93-92
readySWE_0000154 min

Running

49 jobs

AssetWorkerStepProgressWaitedRunning forAttempt
BRA_00001climate-metrics-worker-7b6cddc9fc-6sbnvPHYSICAL
2/6
56 min2 min 11 s1
BEL_00025climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
56 min1 min 60 s1
ESP_00003climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
56 min1 min 56 s1
GBR_00013climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
56 min1 min 53 s1
HUN_00001climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
56 min1 min 48 s1
AUS_00003climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
56 min1 min 44 s1
CHL_00002climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
56 min1 min 42 s1
DEU_00009climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
56 min1 min 38 s1
FRA_00026climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
56 min1 min 35 s1
MLT_00001climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
56 min1 min 32 s1
PRT_00001climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
56 min1 min 28 s1
TWN_00002climate-metrics-worker-7b6cddc9fc-x5zvnstarting
0/6
56 min1 min 27 s1
USA_00086climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
56 min1 min 25 s2
AUS_00004climate-metrics-worker-7b6cddc9fc-x5zvnstarting
0/6
56 min1 min 24 s1
PSE_00001climate-metrics-worker-7b6cddc9fc-8fsxjstarting
0/6
56 min1 min 24 s1
NOR_00001climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
56 min1 min 23 s1
ESP_00009climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
56 min1 min 20 s1
SWE_00001climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min1 min 17 s2
FRA_00020climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min1 min 14 s1
SGP_00028climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min1 min 12 s1
FRA_00030climate-metrics-worker-7b6cddc9fc-2wn4rstarting
0/6
57 min1 min 10 s1
BRA_00003climate-metrics-worker-7b6cddc9fc-6sbnvPHYSICAL
2/6
57 min1 min 8 s1
GUF_00001climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min1 min 7 s1
USA_00014climate-metrics-worker-7b6cddc9fc-x5zvnPHYSICAL
2/6
57 min1 min 7 s1
BRA_00005climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min1 min 3 s1
JPN_00002climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min59 s1
PAN_00001climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min56 s1
VUT_00001climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min53 s1
FRA_00009climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min47 s1
HUN_00001climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min43 s1
AUS_00002climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min38 s1
BRA_00002climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min34 s1
COL_00001climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min31 s1
GBR_00025climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min26 s1
PRT_00001climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min20 s1
USA_00005climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
57 min17 s2
PSE_00001climate-metrics-worker-7b6cddc9fc-7rvrqstarting
0/6
58 min15 s1
VUT_00001climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
58 min14 s2
IND_00002climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
58 min11 s1
ESP_00009climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
58 min7.2 s1
FRA_00017climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
58 min4.8 s1
FRA_00020climate-metrics-worker-7b6cddc9fc-6sbnvstarting
0/6
58 min4.7 s1
FRA_00029climate-metrics-worker-7b6cddc9fc-fd7hrstarting
0/6
58 min4.0 s1
GBR_00027climate-metrics-worker-7b6cddc9fc-2wn4rstarting
0/6
58 min2.9 s1
ITA_00005climate-metrics-worker-7b6cddc9fc-w7kzjstarting
0/6
58 min2.9 s1
SWE_00001climate-metrics-worker-7b6cddc9fc-zxdzwstarting
0/6
58 min1.8 s2
SGP_00028climate-metrics-worker-7b6cddc9fc-x5zvnstarting
0/6
58 min1.7 s1
USA_00005climate-metrics-worker-standby-57c4499f86-g7vrhstarting
0/6
58 min1.6 s1
USA_00017climate-metrics-worker-7b6cddc9fc-v2mxpstarting
0/6
58 min0.3 s1