I'm writing this now, because th…

I’m writing this now, because the reasons for the jubilation below will otherwise be impossible for future generations to understand.

Yesterday (early today really), about 12:45am, we were hit by a web worm that destroyed every html and php file it could find on our server. We were not the only site affected. Our databases were intact, but we had no way of using our tool against such an attack: the panic button. Because the panic button had been destroyed.

What followed was a crisis. I had a backup from Saturday, but was afraid restoring would overwrite the database. I called Kenny (on the phone ) in PA to inform him of the problem. He didn’t have copies of the scripts either. I waited for our host to say something, only to realize every other site had been affected as well, and that it was a brand new worm. Step A in getting Jux somewhat back was replacing the worm-generated index with our own, simplistic one.

Kenny and I then tried to hack into the ENSIM generated backup file to see if we could extract just the php scripts and restore the site that way. Unfortunately, the tarball ENSIM creates is encrypted, and we couldn’t access it. At this point I renamed the database to protect it from being overwritten (I hoped) and attempted to restore from the backup for the first time. Nothing happens.

I then sent the entire backup file to Kenny, who also couldn’t open it in any way. It’s well into the afternoon now. I attempt to restore a second time, after Kenny deleted the entire staff directory. Nothing happens. I go to dinner to try to clear my mind, and hope that our host gets back to us.

When I got back from dinner, our host still hadn’t contacted us about the tickets I sent in. There were a few more attempts to get around using the official restore, and I created a new ticket asking the host to restore from their backup if they could, since I was unable to from ours for unknown reasons. I ask the forum-goers on Ars Technica if they have any idea of how to break the encryption used by ENSIM. No one does.

Finally, I give up and try restoring from our backup a third time. An hour and a half later, after sending the massive tarball somewhere for the fourth time in the past 24 hours, I open the staff directory expecting it to be empty. It wasn’t. The generators were back.

I pressed the panic button as quickly as I could, but several html files that aren’t in the database had been destroyed as well; almost all of these are legacy pages. I uploaded all but three from my hard drive. These three files were ones that I had created on my iBook, which is out for repairs.

Two of those pages were restored by Google Cache, and the third from Archive.org. Thank you, my saviors.

Whew. I know I promised the last feature tonight, but after having been offline for almost an entire day and the work it took to restore this place, I think you’ll understand. The feature will go up tomorrow. Hopefully, we’ll still be here.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s