Understanding Activity and Workflow Task Event Histories in Temporal

Open an event history in the Temporal Web UI and the names are long. Activity Task Scheduled. Activity Task Started. Activity Task Completed. There is a pattern in them, and it is worth a couple of minutes of your attention. Every activity event begins with the same two words. Cross those off, and what is left is a state: Scheduled, in this case. That is the whole convention. It is not a coincidence and it is not occasional. Every activity event in the history is named this way. So here are the five you will meet. Scheduled and Started at the top, then the three ways a task can end: Completed, Failed, and Timed Out. Five names, five states. And those five states are not five separate ideas. They are three steps in one lifecycle, so the next thing to do is lay that lifecycle out.

Those five states are three steps, and an activity task always takes them in the same order. Step one is Scheduled: the Temporal cluster has put the task on a task queue. Nothing of yours has run yet. Step two is Started. A worker has polled that queue, taken the task off it, and begun executing your activity code. Step three is a closed state, and there are three of those. Completed, when the worker ran your code all the way through and returned a value. Failed, when running your code produced an error instead. And Timed Out, when the time allowed ran out before any result came back. So Scheduled is always first, Started is always second, and one of those three is always last. Anything else you see between them in the history belongs to the workflow, not to this task. Once you can see those three steps, the timeout names stop being jargon. Take the one called Start To Close. Its first half names step two, the moment the worker began. Its second half names step three, whichever of those closed states the task reaches. So the timeout is the longest gap allowed between those two moments, and nothing else. A Start To Close of ten seconds says exactly this: once a worker picks the task up, it has ten seconds to finish, to fail, or to be declared timed out. Read another one the same way. Schedule To Start spans step one to step two: how long the task may sit on the queue before a worker picks it up. Every timeout in Temporal is named this way: two points of that lifecycle, and the stretch of time between them.

Now, which of those three closed states a task lands in is decided by two different parties, and the split between them is not arbitrary. Completed and Failed are the worker's call, and that is because the worker is the thing actually running your code. If your activity function returns a value, the worker reports the task as Completed. If it raises an error instead, the worker reports Failed. Same worker, same process, one difference in what your code did. Timed Out is the odd one out. That decision comes from the Temporal cluster, and not from the worker at all. So why is the cluster the one that decides a timeout? Think about what a timeout usually means in a distributed system. Here is the ordinary case. A worker has taken the task and started running your code, and the cluster is waiting for it to report the result back. Now the worker process dies. It cannot finish the task, so no Completed event is coming. It cannot hand you an error, so no Failed event is coming either. It cannot report anything at all. The only party left that can notice is the cluster. It knows when it handed the task out and how long the task was allowed, so when that time is up with nothing received, the cluster writes the timed out event itself. And that is the whole responsibility model. The worker decides Completed or Failed, because it is the only one that watched your code run. The cluster decides Timed Out, because it is the only one still there to notice.

One more thing, and it is the reason any of this is worth learning. The same pattern runs straight through workflow task events. Here are the activity events again, with the state beside each one. And here are the workflow task events. Scheduled, when the cluster puts a workflow task on a queue. Started, when a worker picks it up and runs your workflow code. Then Completed, when the worker is done with it. Failed, when the code raised. And Timed Out, when the cluster stopped waiting. Two different kinds of task, one vocabulary. The state column is the same on both sides, and so is the division of labour: the worker reports completion or failure, and the cluster declares the timeout. So there is nothing here to memorize. Read the suffix on an event, and it tells you where that task had got to. Read the pair of names in a timeout, and it tells you which stretch of that lifecycle the clock was watching.