This page discusses how to debug a Shake-based build system – how to determine why it is going wrong. The page is presented as a series of common problems and how they can be investigated. The common techniques for solving any Shake problem are:
- Switch to single-threaded operation, using
-j1. While concurrency is rarely the cause of the bugs, it often makes the diagnostic messages easier to interpret, and makes reproduction more deterministic.
- Turn on
--lint, which will check for a number of common mistakes.
- Turn up the verbosity level
-vwill print all command lines,
-vvwill print even more, and
-vvvgives a super-detailed commentary on everything that happens (usually for debugging Shake itself).
- If you use
shakeArgsthen every rule can be built individually, e.g.
runhaskell MyShakeFile.hs foo/output_that_fails.txt. Reducing the error can allow higher levels of verbosity.
- If you are still stuck, ask on StackOverflow using the tag
Most Shake errors are formatted as:
Error when running Shake build system: * stack_element1.txt * stack_eLement2.txt * real_file.o Error, rule finished running but did not produce file: real_rile.o
The lines starting with
* provide a stack trace, where
stack_element1.txt depended on
stack_element2.txt, which in turn depended on and ran the rule for
real_file.o. The specific error in this example is that the rule for producing
real_file.o failed to actually create the file - either the file was not created, or was creating in the wrong place.
All Shake errors contain the dependency stack, which is usually the most important thing about diagnosing an error. Often the first step is to map the bottom entry on the stack to the rule that executes to produce it - e.g.
*.o or similar.
Some typical errors you may encounter are:
- recursion detected - when a build system, through a series of dependencies, ends up depending on itself – usually the stack makes it clear where the surprise is. Occasionally you will get an error about indirect recursion, in which case rebuild with
-j1and it will often give a better stack trace.
- file does not exist and no rule available - occurs when you have done
needon a file such as
file.o, but there is no pattern that matches (e.g. no
*.orule), and the file is not a source file (which do not need explicit patterns). Usual solution is to add a pattern that builds the artefact.
- key matches multiple rules - occurs when you multiple patterns that match a single file, which is not allowed - try making your rules more specific so they are disjoint.
- system command failed - occurs when you run a system command using
cmd, not capturing the
ExitCode, and it either fails to start or returns a non-zero exit code. The next line should contain the system command and arguments.
Since build systems typically spend most of their time calling out to command line tools, much of the debugging effort is around getting these tools to behave properly. The easiest way to debug a system command is to avoid it entirely - tools like
sed can usually be replaced with Haskell code, which can fix issues and improve portability.
The first step to debugging a command line tool is to reproduce the error outside of Shake, typically by passing
-v to see the command line and pasting it into a shell. Unfortunately all the platform abstractions can sometimes get in the way, in particular pay attention to:
- Quoting, especially on Windows, where the conventions are difficult to fathom.
- What the command is wrapped in - specifically
Shellcauses the command to be run through
cmd, anything such as
stack execcan also add translation layers.
- Environment issues such as current directory and environment variables.
- Any output flags, such as
-o, to control where the output goes.
- Precise details - things like using a short flag vs long flag, ordering of flags etc. can all have an impact.
Alas, each tool is different, and all present unique challenges. Where possible copy the command lines from other systems, rather than trying to rediscover this arcane knowledge.
See the HTML Profile Reports, which provide overviews of dependencies. These can be mined to figure out the causes of rebuilding.