[Yocto-infrastructure] Follow up: Outage Friday, March 29 2013, 1900 - 2000 UTC

[Michael Halstead]

The power upgrade was completed at 20:20 UTC (1:20pm PDT). We now have
room to add additional builders as required. We ran over our outage
window by 20 minutes do to switch vlan configuration not loading
properly after a power cycle. This has been corrected and the switches
now come up properly after a full cycle.

The downtime revealed an unrecoverable filesystem error on AB06. AB06
was the only builder running without ECC enabled. This was to test the
impact on performance when filling all available DIMM slots. One of the
DIMMs was reported faulty after a memory test. The damaged module has
been removed and ECC has been enabled on the server. I will reinstall
the operating system by the end of Monday. These errors will be
prevented in the future by running with ECC enabled.

Michael Halstead
Yocto Project / Sys Admin

On 03/27/2013 03:26 PM, Michael Halstead wrote:
> We are upgrading our power distribution units in order to add additional
> servers in the future. To make the upgrade as easy and safe as possible
> we will take down all of the Yocto Project servers. We will also take
> the opportunity to apply needed firmware upgrades. The outage should
> last 45 minutes.
>
> Service(s) affected: All Yocto Project services will be interrupted including the website, lists, git, wiki, and access to autobuilders.
>
> Outage window: Friday, March 29 2013, 1900 - 2000 UTC
> 		Friday, March 29 2013, 12:00 noon - 1:00pm PDT
>
>


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4516 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.yoctoproject.org/pipermail/yocto-infrastructure/attachments/20130329/5d7a97fc/attachment.bin>