[yocto] [AB PATCH 00/27] Autobuilder patches
Elizabeth Flanagan
elizabeth.flanagan at intel.com
Wed Mar 5 10:22:51 PST 2014
This is a fairly large patchset and I'd like at least a second set of eyes on it.
There are a few things this does:
1. Updates the Autobuilder parser.
The autobuilder parser was fairly junky. It didn't dive too far into what it was
parsing. This made it so people couldn't do triggers of triggers because the
props wouldn't filter all the way up to the master trigger. This should be
possible now.
2. Simplify the commit/repo/branch combos/remove mix and match.
A long time ago we needed the ability to mix and match repos/branches when we
were building out nightly. This made it so that we could build everything but
nightly-foo on master. That use case is no longer really needed.
It was also a major complaint that the UI was complicated. This trims the UI
down and makes it much sparser
3. Git fetcher mirror.
The git fetcher with buildbot doesn't do a lot of what we need it to. This set
of patches creates a mirror on each slave of the major repos. This allows us to
clone from those repos. The git fetcher also has a movecopy method that allows
us to instead of clobbering a build dir, move it off to a trash directory. This
works well when your trash and slaves are on the same partition but is slowed
when you move it across disk.
When the ab starts a service runs in the background that 1. Updates all of the
repos every XX minutes 2. takes out the trash every XX minutes.
The following changes since commit ba57054aea5627b557327632ad3d6c2a1faa14f2:
4. Simplifies wait/no_wait triggers.
This needs to be revisited, as I know of a better way to do this now, but this
allows us to more optimally utilize the ab resources by triggering all builds,
but putting a few second sleep in for the no_wait builds to queue.
Bugs fixed:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=5454
https://bugzilla.yoctoproject.org/show_bug.cgi?id=5433
https://bugzilla.yoctoproject.org/show_bug.cgi?id=5027
PublishLayerTarballs: Don't hang is PUBLISH_BUILDS isn't set (2014-02-18 10:37:36 +0000)
are available in the git repository at:
git://git.yoctoproject.org/yocto-autobuilder eflanagan/yocto-autobuilder-dev-rebase
http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/log/?h=eflanagan/yocto-autobuilder-dev-rebase
Beth Flanagan (8):
PrepPkgIndex.py: ln instead of cp
Autobuilder.py/Buildset.py: More sane parser/UI
nightly.conf: Add additional parameters
CheckOutLayers.py/ResolveLayerHead.py: support simplified git
Autobuilder.py: remove DEBUG code
BuildSet.py: Remove more debug code.
yocto-start/stop-autobuilder: start buildslave-janitor
forms.html/layout.html: support simplified UI
Elizabeth Flanagan (12):
nightly-world-uclibc: uclibc target
SyncPersistDB.py: First run failure correction
BuildSet.py: Set workdir based on layer not order
BuildSet.py: remove trailing spaces
yoctogit.py: fix callbacks and instance attrs
PublishArtifacts.py: Fix poky-tiny publish
PublishArtifacts.py: genericx86-64 publish
BuildEclipsePlugin.py: Change location of build.
PublishArtifacts.py: Change where we find eclipse artifacts
yocto-autobuilder-setup: set the correct bb version
fs.py: Bump default timeout
autobuilder: whitespace removal
Richard Purdie (5):
Add mirror functionality to git fetcher
trigger.py: Allow wait and nowait triggers
yoctogit: Don't use branch args for referenced clone
bin/buildslave-janitor: For fast-git checkouts
yoctogit: Remove use of oe.path.exists()
Stefan Stanacar (2):
buildsets: add a new buildset to run builds on target
Add buildstep and buildset for oe-selftest script
bin/buildslave-janitor | 125 ++++++
buildset-config.master/nightly-oe-selftest.conf | 15 +
.../nightly-qa-targetbuilds.conf | 26 ++
buildset-config.master/nightly-world-uclibc.conf | 18 +
buildset-config.master/nightly.conf | 40 +-
config/autobuilder.conf.example | 1 +
.../site-packages/autobuilder/Autobuilder.py | 422 ++++++++++-----------
.../site-packages/autobuilder/BuildSet.py | 67 ++--
.../autobuilder/buildsteps/BuildEclipsePlugin.py | 8 +-
.../autobuilder/buildsteps/BuildToolchainImages.py | 6 +-
.../autobuilder/buildsteps/CheckOutLayers.py | 32 +-
.../autobuilder/buildsteps/CreateAutoConf.py | 17 +-
.../autobuilder/buildsteps/CreateBBLayersConf.py | 10 +-
.../autobuilder/buildsteps/CreateCurrentLink.py | 6 +-
.../buildsteps/CreateIntelBSPPackage.py | 8 +-
.../autobuilder/buildsteps/GetDistroVersion.py | 4 +-
.../autobuilder/buildsteps/GetLayerVersion.py | 10 +-
.../autobuilder/buildsteps/HelloWorld.py | 6 +-
.../autobuilder/buildsteps/MaintainPersistDB.py | 12 +-
.../autobuilder/buildsteps/PrepPkgIndex.py | 12 +-
.../autobuilder/buildsteps/PublishArtifacts.py | 28 +-
.../autobuilder/buildsteps/PublishLayerTarballs.py | 33 +-
.../autobuilder/buildsteps/ResolveLayerHead.py | 32 +-
.../autobuilder/buildsteps/RunOeSelftest.py | 26 ++
.../autobuilder/buildsteps/RunSanityTests.py | 6 +-
.../autobuilder/buildsteps/SetDest.py | 12 +-
.../site-packages/autobuilder/buildsteps/Sleep.py | 20 +-
.../autobuilder/buildsteps/SyncPersistDB.py | 11 +-
.../autobuilder/buildsteps/TestFailStep.py | 4 +-
.../autobuilder/buildsteps/YoctoBlocker.py | 2 +-
.../buildbot/status/web/templates/forms.html | 1 -
.../buildbot/status/web/templates/layout.html | 185 +--------
.../buildbot/steps/source/yoctogit.py | 99 +++--
.../buildbot/steps/trigger.py | 15 +-
.../buildslave/commands/fs.py | 4 +-
yocto-autobuilder-setup | 4 +-
yocto-start-autobuilder | 24 +-
yocto-stop-autobuilder | 41 +-
38 files changed, 756 insertions(+), 636 deletions(-)
create mode 100755 bin/buildslave-janitor
create mode 100644 buildset-config.master/nightly-oe-selftest.conf
create mode 100644 buildset-config.master/nightly-qa-targetbuilds.conf
create mode 100644 buildset-config.master/nightly-world-uclibc.conf
create mode 100644 lib/python2.7/site-packages/autobuilder/buildsteps/RunOeSelftest.py
--
1.8.1.2
More information about the yocto
mailing list