Scythe - Coverage in Production to Find Dead Code
A while back I wrote about what it takes to kill code. I won’t take up any time repeating why it’s a good idea to get rid of code that’s dead or low value. I think that anyone who’s spent time trying to understand or change some area of code only to find out that it’s never really used knows how frustrating it can be.
As often as this happens, we just tend to live with dead code.
It seems like it should be easy to find code that doesn’t execute, but aside from static analysis tools - which show code that logically can’t execute - all we have to fall back on are logs and telemetry. If we are lucky, we have a run time that can give us notification when methods are called, but that can be costly. It presents the issue we’d have if we chose to run test coverage tools in production - it’s slow. It would be impractical to run coverage all of the time.
Considering all of this, I decided to create a tool that automates something I’d been doing for a while by hand: inserting probes in areas of code that I suspect are dead and checking periodically to see if they’ve been called.
The tool is called Scythe and it’s very simple to use. You place calls that look like this in your source.
scythe_probe(“margin_overrun_case”);
The name margin_overrun_case is a probe name. You don’t have to declare these names anyplace. Just having a scythe_probe call with that string as an argument is enough.
After you've inserted a probe, run Scythe to have it scan all of your source files and record any probe names that it hasn’t seen before. Ideally, you do this as part of your build.
When your application runs, the probes record the time and date that they are called. You can run Scythe on the command line at any time to see the number of seconds, hours, or days since the last call of each probe.
That’s it. Simple.
Right now, Scythe supports Java, Python 3, and Ruby. Adding other languages is easy. It’s just a matter of adding a single function.
I tend to believe that flexibility comes from what you leave out rather than what you add, so I’ve made Scythe maximally flexible by giving it an extremely simple implementation. Each probe is recorded as a zero-length file in a directory you specify through an environment variable. When Scythe scans sources it creates a file for any probes it hasn’t seen before. When scythe_probe is called, all it does is touch the file for the probe to update its modification time. This is near zero cost, and it will work for all languages and all platforms with a file system.
Try it out. If you suspect that some area of your code is never called or is called extremely infrequently, insert a probe. Check periodically to see if it’s been hit.
Feel free to add support for other languages.