10:03 p.m.
Hi,
The x86_64 builder (ciall.foxkit.us) has had a catastrophic disk failure
at around 23:44 CST 18 February 2018. During attempted recovery, there
was a power failure in the Tulsa metro at 04:14:47 CST 19 February 2018
that lasted over two hours; regrettably, much longer than the 45 minute
backup power runtime. Due to these events occuring so close together,
the entire /srv (package staging area), /usr/src (aports git trees that
had a few uncommitted patches), and buildroot / (where abuild
configuration lived) trees are completely unrecoverable.
The /opt/virtual tree, where the Adélie Portability Virtual Machines
lived (including NetBSD-based bryan.foxkit.us, OpenBSD-based
georgie.foxkit.us, and Arch-based fran.foxkit.us), may be recoverable -
it is not yet known at this time.
Anyone who had any data on bryan, georgie, or fran should consider that
data gone barring an exceptional recovery effort. As their primary
purpose was for portability testing only, no backups were ever taken of
their virtual disks.
The /home tree on ciall was in the process of its monthly back up at the
time of the disk failure and it appears only a few files (from my own ~)
were lost. No other user data was lost.
As x86_64 is a tier 1 architecture, and the Platform Team is not
currently considering deprecating it, this unexpected, catastrophic,
poorly-timed sequence of failures may cause the alpha5 release date to
slip again. This is regretful and unfortunate. Moving forward, we will
need to discuss if architecture buildroots and configurations should be
stored in Git; if a Git repository isn't deemed a proper solution, they
will at least be added to the monthly back up jobs.
Sincerely,
--arw
--
A. Wilcox (awilfox)
Project Lead, Adélie Linux
http://adelielinux.org