project-config/jenkins
Ian Wienand 8928d5f626 Run wheel build jobs in parallel and keep logs
Even after increasing the build-time to 180 minutes with
Idc1d53d1e73b4a9eb13f258f4a4d5627ec3cf300 it's not enough to avoid
timeouts.  I did a manual run and it took 214 minutes on the centos
builder.

We can parallelise builds.  The "parallel" program provides us with
plenty of useful help in this regard.  As described in the comments,
this will run ncpu jobs (4 on our builders) and capture and store the
logs for each individual job.  A test run on the centos build host was
reduced to ~1 hour.

The job output is particularly handy because we can do a little work
to make an easy-to-parse failure log.  This job is a little weird
because failures aren't hard -- we can still release everything we did
manage to build.  Initially this overview log file will enable a 3rd
party monitor that can alert interested people when failing builds
occur.

We store the build logs as a .tar.bz2 bundle as described, but the
smaller files are copied directly.

Change-Id: Ifec15f71cc3530d47da4f3304d3fe094d77e0980
Depends-On: I9635721a4f8d718ad402d23600840f091267952c
2017-01-24 17:10:42 +11:00
..
data Fix bindep-fallback.txt for fedora 2017-01-14 11:21:28 -05:00
jobs Run wheel build jobs in parallel and keep logs 2017-01-24 17:10:42 +11:00
modules/jjb_afs Add JJB AFS module 2016-09-27 14:49:09 -07:00
scripts Run wheel build jobs in parallel and keep logs 2017-01-24 17:10:42 +11:00