FAQ

Q: What else is on this website?

Q: Any more documentation?

There is a complete list of every function in Shake which can be searched. Each function comes with documentation and examples.

Much of the theory behind Shake is covered in a conference paper which was accompanied by this video (slides). Since then I've given videoed talks on small worked examples (slides) and how to structure large Shake systems (slides).

I sometimes write about ongoing development work or other Shake-related things on my blog.

If you have any further questions:

Q: Is Shake limited to building Haskell?

Not at all – Shake can build any project in any combination of languages. In fact, Shake isn't typically necessary for vanilla Haskell projects, as you can use cabal or stack. Shake is often used for building C/C++, Docker containers and Javascript/HTML/CSS projects.

Q: Where are functions for string manipulation?

Shake is a Haskell package focused on providing build-system functionality. Since Shake scripts are written in Haskell, they can easily access other Haskell packages. Most general needs are met by the standard base library, but a few other useful general functions can be found in the extra library (e.g. trim and replace). For more specific functionality (e.g. parsing, databases, JSON) find a suitable Haskell library and use that.

Q: Why is there a shake executable?

Most users will write their own Haskell file and compile it to produce an executable that is their build tool. The shake executable is there to run the demo, run Ninja build files and will also run a Shakefile.hs if present.

Q: Can file patterns overlap?

No. If two patterns overlap for a file being built it will result in a runtime error – you cannot have a pattern for *.txt, and another for foo.*, and then build a file named foo.txt. For objects that typically share the same extension (e.g. C and Haskell both produce .o objects), either disambiguate with a different extension (e.g. .c.o and .hs.o), or different directory (e.g. obj/c/**/.o and obj/hs/**/.o). For more information, including ways to enable overlap and set priorities, see %>.

Q: Why do multiple calls to need run sequentially? Are Applicative actions run in parallel?

In Shake, need xs >> need ys will build xs in parallel, then afterwards build ys in parallel. The same is true of need xs *> need ys, where *> is the applicative equivalent of >>. In contrast, Haxl will execute both arguments to *> in parallel. For Shake, you are encouraged to merge adjacent need operations (e.g. need (xs++ys)), and where that is not possible (e.g. when using askOracle) use parallel explicitly.

Shake could follow the Haxl approach, but does not, mainly because they are targeting different problems. In Haxl, the operations are typically read-only, and any single step is likely to involve lots of operations. In contrast, with Shake the operations definitely change the file system, and there are typically only one or two per rule. Consequently, Shake opts for an explicit approach, rather than allow users to use *> (and then inevitably add a comment because its an unusual thing to do).

Q: Should file names be relative or absolute?

We recommend using only relative file names in Shake.

Shake can use file names that are either absolute (C:\file.txt, /file.txt) or relative (file.txt). However, Shake compares filenames by value, so if you need both file.txt and /file.txt it will attempt to build both, likely resulting in failure - within a single Shake project you must stick to either relative or absolute file names.

Q: How can I depend on directories?

Think of directories as containers for files. They exist or don't pretty randomly, but if they have files, they must exist. In particular, you can't depend on a directory with need or write a rule to create a directory. Directories are created as needed – the rule for bar/baz.exe will create the bar directory if necessary. If you want to depend on a git clone having being performed, depend on a particular checked-out file instead (e.g. README.md), with the rule to create it being git clone.

There is a tracked function doesDirectoryExist, to depend on the presence or absence of a directory, but you should not call it on directories which might be created by the build system.

Q: What's the history of Shake?

I (Neil Mitchell) was one of the people behind the Yhc project, a Haskell compiler that died in a large part because of its build system. To quote from the final blog post:

The biggest challenge for Yhc was the build system – we ended up with 10,000 lines of Python Scons scripts. Without a robust build system nothing else matters. When our sole Python hacker left the team that was the beginning of the end.

A Haskell compiler is a big undertaking, but the build system for a simple Haskell compiler shouldn't be that complicated.

When writing my thesis I needed a build system, and decided to try writing a simple Haskell DSL, which is still online here. I defined a single operator <== which let me express a relationship between an output and its dependencies – very simple, but it worked.

Later I moved to Standard Chartered, where the build system was a mass of Makefiles, and it quickly became apparent that the project had outgrown the current approach. Without really surveying the alternatives, I decided that a Haskell DSL would be easiest to fit in with the existing infrastructure, so started writing some code. The first version of the build library took under a week, followed by a month of reimplementing the existing system. It wasn't until many months later I realised that the reason everything was suddenly so much easier was because we had monadic dependencies.

While people at Standard Chartered wanted to open source Shake, that turned out not to be possible. A number of people in the Haskell community implemented their own versions of Shake, but none were as polished or as strong as our internal one. Eventually, I reimplemented Shake, from scratch, in my spare time. Writing Shake from scratch, without the original source code or documentation, it naturally turned out better than the first attempt. A while later Standard Chartered migrated to the open-source version.

I still maintain Shake, but am fortunate to have other contributors extending and improving Shake. If you want to join in, see notes for developers.