Kevin Svetlitski
52b9b4863f
The existing `Trace_writer` module works well enough (albeit not perfectly) most of the time. However, it is difficult to reason about, in large part because it writes the trace in a streaming fashion. That introduces significant additional complexity and bookkeeping, and limits the ability of the trace-writer to make use of information discovered later in the trace (I believe the latter is why traces produced today often have the few frames closest to the root wrong). Because we want to extend the trace-writer with new functionality, we're starting fresh with a different design that's easier to reason about. The new implementation currently exists alongside the original, but the goal is to eventually replace it entirely. Instead of writing the trace in a streaming fashion, we construct an internal representation of the trace in memory, and write out the trace in a separate, final pass once all of the events have been consumed. The module responsible for doing most of the heavy lifting is the new `Trace_segment`, which represents a continuous, lossless, and error-free segment of the trace; we create a new trace-segment whenever we encounter an error. The other major addition is that the new implementation includes inlined function calls, using LLVM for symbolization, dramatically increasing the fidelity of the trace. **This PR is effectively an alpha of the new implementation.** The code here does indeed work, and produces better traces than the existing backend in many cases, but there are a couple critical issues: 1. **Error recovery**: We create a new trace-segment whenever we encounter an error, **but at present we naively treat each trace-segment as disjoint**. We need to add an additional "stitching" pass before the trace is written out, making a heuristic, best-effort attempt to join together adjacent trace-segments in a way that preserves control-flow continuity. All of this is a long way of saying that if you encounter *any* error while using the new implementation, your trace is likely to be horribly broken. 2. **Performance**: Including inlined frames makes the traces significantly larger, and the supporting code for this new functionality is written pretty naively from a performance standpoint. As a result, the new implementation is roughly 2x slower than the old implementation. It should also go without saying that while this code appears to work well on the traces I've tried it on, I would not at all be surprised if there are still bugs/edge-cases. We will address these shortcomings over time, but in the meantime the new implementation is opt-in; setting the environment variable `MAGIC_TRACE_USE_NEW_TRACE_WRITER=1` will enable it. Signed-off-by: Kevin Svetlitski <ksvetlitski@janestreet.com>
Magic_trace library
Used with the magic-trace tool to capture a trace of the execution
leading up to a call to Magic_trace.take_snapshot () for performance
analysis and debugging.
The tool can attach to any symbol, but it defaults to the special symbol used by this library, making this the easiest way to modify your code to take a snapshot under certain conditions.