This reminded me of logging protocols in CS145 -- for instance a Write ahead log allows us to reconstruct transactions that did not execute from reading a log of what was supposed to have been executed (That is written before the actual transactions are executed). This feels similar in that we need to maintain a log of transformations that constitute a lineage so that upon failure during the lineage of transformations, we can just read the log and re execute them in the same order
This reminded me of logging protocols in CS145 -- for instance a Write ahead log allows us to reconstruct transactions that did not execute from reading a log of what was supposed to have been executed (That is written before the actual transactions are executed). This feels similar in that we need to maintain a log of transformations that constitute a lineage so that upon failure during the lineage of transformations, we can just read the log and re execute them in the same order