=== once: HANG_ON=T3 HANG_TIMES=1 ===
[watchdog] start #1: worker pid 226
worker pid=226 start T1 parse input files
worker: done T1
worker pid=226 start T2 index the records
worker: done T2
worker pid=226 start T3 summarise findings
worker: HANGING on T3
[watchdog] no heartbeat for 2.0s -> killing worker pid 226 and its process group
[watchdog] T3 hung 1x -> will retry
[watchdog] start #2: worker pid 230
worker pid=230 start T3 summarise findings
worker: done T3
worker pid=230 start T4 draft the report
worker: done T4
worker pid=230 start T5 archive outputs
worker: done T5
[watchdog] worker exited with 0
[watchdog] finished; all tasks resolved
watchdog exit code: 0
--- final list
T1  done     hangs=0  parse input files
T2  done     hangs=0  index the records
T3  done     hangs=1  summarise findings
T4  done     hangs=0  draft the report
T5  done     hangs=0  archive outputs
--- leftover processes:
none

=== always: HANG_ON=T3 HANG_TIMES=99 ===
[watchdog] start #1: worker pid 242
worker pid=242 start T1 parse input files
worker: done T1
worker pid=242 start T2 index the records
worker: done T2
worker pid=242 start T3 summarise findings
worker: HANGING on T3
[watchdog] no heartbeat for 2.0s -> killing worker pid 242 and its process group
[watchdog] T3 hung 1x -> will retry
[watchdog] start #2: worker pid 246
worker pid=246 start T3 summarise findings
worker: HANGING on T3
[watchdog] no heartbeat for 2.0s -> killing worker pid 246 and its process group
[watchdog] T3 hung 2x -> set aside
[watchdog] start #3: worker pid 248
worker pid=248 start T4 draft the report
worker: done T4
worker pid=248 start T5 archive outputs
worker: done T5
[watchdog] worker exited with 0
[watchdog] finished; all tasks resolved
watchdog exit code: 0
--- final list
T1  done     hangs=0  parse input files
T2  done     hangs=0  index the records
T3  skipped  hangs=2  summarise findings
T4  done     hangs=0  draft the report
T5  done     hangs=0  archive outputs
--- leftover processes:
none

=== give-up: HANG_ON=* HANG_TIMES=99 ===
[watchdog] start #1: worker pid 259
worker pid=259 start T1 parse input files
worker: HANGING on T1
[watchdog] no heartbeat for 2.0s -> killing worker pid 259 and its process group
[watchdog] T1 hung 1x -> will retry
[watchdog] start #2: worker pid 261
worker pid=261 start T1 parse input files
worker: HANGING on T1
[watchdog] no heartbeat for 2.0s -> killing worker pid 261 and its process group
[watchdog] T1 hung 2x -> set aside
[watchdog] start #3: worker pid 270
worker pid=270 start T2 index the records
worker: HANGING on T2
[watchdog] no heartbeat for 2.0s -> killing worker pid 270 and its process group
[watchdog] T2 hung 1x -> will retry
[watchdog] start #4: worker pid 272
worker pid=272 start T2 index the records
worker: HANGING on T2
[watchdog] no heartbeat for 2.0s -> killing worker pid 272 and its process group
[watchdog] T2 hung 2x -> set aside
[watchdog] start #5: worker pid 274
worker pid=274 start T3 summarise findings
worker: HANGING on T3
[watchdog] no heartbeat for 2.0s -> killing worker pid 274 and its process group
[watchdog] T3 hung 1x -> will retry
[watchdog] giving up after 5 starts
watchdog exit code: 1
--- final list
T1  skipped  hangs=2  parse input files
T2  skipped  hangs=2  index the records
T3  pending  hangs=1  summarise findings
T4  pending  hangs=0  draft the report
T5  pending  hangs=0  archive outputs
--- leftover processes:
none

$ final process check
none
