[Nix-dev] enableParallelbuilding for many packages
Marc Weber
marco-oweber at gmx.de
Wed Oct 24 23:28:20 CEST 2012
I've written a small quick and dirty ruby script:
http://mawercer.de/run-tests.rb
for each package if enableParallelbuilding is not set yet it will build
it three times:
1st to build all dependencies (with enableParallelbuilding)
2nd without enableParallelbuilding
3rd with enableParallelbuilding
If all 3 succeed this means that building with enableParallelbuilding
succeeded two times the build times are compared.
If enableParallelbuilding is significant faster (more than 20%)
better is set to true
Some results are already available here:
http://mawercer.de/cache
Thus there are already 104 packages which might benefit significantly
from being built in parallel.
When the script is done (which may take some more days because my
v-server only has 4 cores) I'll turn this into a patch.
This only tests x86_64 builds.
I also have to say that there are some builds which randomly fail
rarely. (Eg ruby did once "make internal error", but I couldn't
reproduce it doing 5 additional builds)
Thus I'm even thinking about running up to two or three times in such
cases - or to patch nix to redo the whole build up to x times if a
regular expressions such as make token errors are encountered in the
logs.
People who think accuracy is most important will disable this feature
anyway - but some like me sometimes just have to do rebuilds .. and
they may just work more often than not.
If you have any objections about such a patch adding many
enableParallelbuilding = true; lines based on my results you should tell
me and make me understand why.
Marc Weber
More information about the nix-dev
mailing list