[Nix-dev] How do parallel builds scale?

[Marc Weber]

Hi ludo,

I've seen your post (autoconf mailinglist) asking about how to scale
configure scripts. I don't think there is an easy way. :(
Maybe using cmake could be an option.

More interesting: Have you also numbers about whether some of those
builds failed due to -jX ?

That was one of the main arguments against a feature like
enableParallelBuilding in the past (?). And discussion stopped at
exactly this point.

I've seen an internal make failure for some packages ocassionally. Eg
ruby19:
  INTERNAL: Exiting with 6 jobserver tokens available; should be 4!

> The build phase itself scales better, but for half of the packages the
> speedup is lower than 4.

Many projects (eg HaXe or terraview) just don't care enough about it for
whatever reason.

I've had another idea in mind: Building up a database listing the
resources a build takes (both IO and CPU). Then it would be possible to
calculate an estimated wait time for rebuilds - or maybe it would be
possible to optimize the order of builds or such.

I don't care enough to work on it right now.

Marc Weber