[Nix-dev] enableParallelbuilding for many packages

Marc Weber marco-oweber at gmx.de
Wed Oct 24 23:28:20 CEST 2012


I've written a small quick and dirty ruby script:
http://mawercer.de/run-tests.rb

for each package if enableParallelbuilding is not set yet it will build
it three times:

1st to build all dependencies (with enableParallelbuilding)
2nd without enableParallelbuilding
3rd with enableParallelbuilding

If all 3 succeed this means that building with enableParallelbuilding
succeeded two times the build times are compared.

If enableParallelbuilding is significant faster (more than 20%)
better is set to true

Some results are already available here:
http://mawercer.de/cache

Thus there are already 104 packages which might benefit significantly
from being built in parallel.

When the script is done (which may take some more days because my
v-server only has 4 cores) I'll turn this into a patch.

This only tests x86_64 builds.

I also have to say that there are some builds which randomly fail
rarely. (Eg ruby did once "make internal error", but I couldn't
reproduce it doing 5 additional builds)

Thus I'm even thinking about running up to two or three times in such
cases - or to patch nix to redo the whole build up to x times if a
regular expressions such as make token errors are encountered in the
logs.

People who think accuracy is most important will disable this feature
anyway - but some like me sometimes just have to do rebuilds .. and 
they may just work more often than not.

If you have any objections about such a patch adding many
enableParallelbuilding = true; lines based on my results you should tell
me and make me understand why.

Marc Weber


More information about the nix-dev mailing list