[NTLK] 2018-07-05 UNNA.org outage status update

[L.W. Brown]

All the very best—wish I could provide help

-L.W. Brown
via 6s+

> On Jul 5, 2018, at 09:52, Morgan Aldridge <morgant at makkintosshu.com> wrote:
> 
> Hey all,
> 
> First, my sincerest apologies for the long downtime for UNNA.org site and
> the lack of communication. I have recovered the UNNA data and am currently
> rebuilding infrastructure to so I can start putting the pieces back
> together. This outage is affecting my own small business in innumerable
> ways, so it's extremely high priority and is my sole focus right now.
> 
> Timeline of Events:
> 
> 2018-06-23 - 2018-06-26: The web server hosting UNNA.org, plus a number of
> other sites hosted on makkintosshu.net crashed sometime early Sunday
> morning. Unfortunately, I was out of town on a canoe camping trip without
> cell or Internet service, and didn't notice until Tuesday morning when I
> finally got back to checking email.
> 
> 2018-06-29 - Present: The aforementioned server crashed again sometime
> early Friday morning, this time resulting in catastrophic, unrepairable
> data corruption. The first few days were spent in data recovery, which
> appears to have been successful, plus planning & prepping for upgrades,
> service & data migrations, and required reconfigurations. Unfortunately,
> some of the additional services the server was running resulted in
> cascading failures of makkintosshu.net network & server infrastructure, so
> I've since been rebuilding & troubleshooting that so I can get websites
> back up and running (and get back to now significantly delayed client work).
> 
> Going Forward:
> 
> Once infrastructure & web servers are restored, there will actually be a
> fair amount of additional work to get UNNA.org back up and running again,
> roughly:
> 
> 1) UNNA data is hosted in a large Subversion repository which includes the
> websites (www.unna.org, tools.unna.org, mirrors.unna.org, etc.), but this
> is slow and unwieldy. That will need to be migrated to separate git
> repositories (they'll end up under the UNNA GitHub organization: <
> https://github.com/unna/>) for several reasons: performance, manageability,
> additional backups, improved community access/support, and easier
> restoration of UNNA mirrors.
> 2) Then I should be able to get the UNNA.org sites back up and running
> relatively quickly.
> 4) There will be some code that needs to be updated to restore all
> functionality, such as automatic commits when files are uploaded/published
> (currently SVN), background processes, etc.
> 3) Next comes restoration (and updates) of additional services such as UNNA
> torrents seeds, NPDS tracker, etc.
> 5) Finally, implementing & documenting a new process of hosting UNNA
> mirrors, utilizing the Git repos (previous versions used rsync, which is
> great, but using version control—esp. git—will be much easier & faster).
> 6) ...
> 7) Non-profit
> 
> That's just the UNNA punch list, and I'm already nearly a week into
> infrastructure rebuild with all too many "bang head here" moments, plus
> client sites and contract work to get back to ASAP. I greatly appreciate
> everyone's patience so far, I'm sure you're all fighting back many anxious
> questions. I'll try to answer questions & provide more timely updates, but
> it'll be very much as time permits.
> 
> I know several of your have expressed interest in hosting a mirror over the
> years. Let's discuss once the data is up on GitHub, but probably no sooner.
> At that point, I'll also happy to discuss other improvements, upgrades, etc.
> 
> Wishing for cool & calm,
> 
> Morgan
> -- 
> Morgan
> ---
> http://makkintosshu.com/
> http://centresteer.com/
> http://seriesparts.com/
> http://unna.org/
> ----------------------------------------------------------------------
> 
> http://newtontalk.net/
> http://twitter.com/newtontalk