Tag: Security

Ready for 2010: Check Your Backups

As usual, this is something you should to at least once each month or each week – go into your backups and check that they’re actually running and doing what they should be doing. If you think you’re doing backups – but you’re unable to restore from the backup, you’re not doing backups!

As Jeff Atwood can attest: DO NOT TRUST THAT OTHER PEOPLE ARE DOING YOUR BACKUPS – not even your hosting provider even if you pay them to do this for you. At least something good came out of that story – I think quite a few folks checked their own backups in the days after the disaster happened. The only person who really has any great interest in your backup working is YOU. Sure, people may lose your business because of ruining your backups (and they should have a great interest in making their backups work), but the person who’ll be greatly affect the loss of your backup will be you (and other people caught in the same shitstorm).

So here’s a couple of quick things you could do to maybe do things a bit better than last year:

  1. Revisit all backup-scripts and run them manually to check that they actually work. Maybe your SSH-keys have expired (debian blacklisted SSH-keys generated with a bogus key generation function) (been there, done that), your SSH-daemon has switched to another port and you’ve not updated your scripts (been there, done that) or several other causes. Run them manully first, so that you’re catching any errors. It is not enough to think that cron (or Scheduled Tasks) would tell you if anything weird happened.
  2. Make your backups travel out from the physical location of the server. This might be your house, a co-location facility, your own dedicated server room, etc., but it will never be guarded against an attack by the natural forces such as a large fire, a flood, a train or an airplane hitting the building. Get it out of the building, and get it out of the city. Purchase online backup storage if needed, it’s dirt cheap at the moment and will still be in the future. $5-$20 a month for keeping all your data safe? It’s not even worth to spend any time thinking about it. Get it running, NOW!
  3. Check that there’s enough free disk space on the server you’re making your backups to. Doesn’t help if everything works today if you’re going to run out of space in 6 days. Make the server send you a notice through E-mail and SMS if it’s getting anyewhere close to 80% of total capacity. Install munin, nagios and/or cacti to keep track of your servers. I’ve become a fan of the easiness of munin, so if you haven’t taken a closer look at it, do it now.
  4. Check that your warning systems when a backup fails actually work. You could create a small file in the location you’re backing up with random content and then checking the md5sums of the files after the backup job has run. Make the alarm bells go off when things doesn’t match.
  5. Make a static backup on an USB drive. USB-drives have also become dirt cheap, get a 1TB one and make a backup right now. Store the drive somewhere else than your regular backups, such as your home. Run the backup each month. This will at least make you do the backup manually and might save your ass when you discover that your regular backup job didn’t backup what you thought it backed up.
  6. Backup jobs only back up what you tell them to back up. If you’re not backing up a complete system (and be sure to check that the backup includes files that might be kept open by the system, such as the hard drive image files of virtualized servers), you might have installed new applications that contain user generated files, new database storage locations etc, during the year. Be sure to check that you actually back up the things that are worth something to you. Maybe your new image application stores it metadata in another location than your previous one (Lightroom vs Aperture perhaps). Check it.
  7. Do NOT write an article about backup routines and backup suggestions. This will result in catastrophic hardrive failure and backups that don’t work. You WILL jinx yourself. Oh sod.

If you need a good suggestion for backup software, take a look at rdiff-backup.

I have probably forgotten something very important here, so I’m trusting you guys to be my backup (yeah, right). Leave a comment with any good advice!

Now go out and create a random file somewhere on the locations you’re backing up.

Read all the articles in the Ready for 2010-series.

A Redirect Does Not Stop Execution

This is just a public service announcement for all the inexperienced developers who are writing redirects in PHP by issuing a call to header(“Location: <new url>”) to do their redirect. I see the same mistake time over and over again, and just to try to make sure that people actually remember this:

A Call to Header (even with Location:) Does NOT Stop The Execution of the Current Application!

A simple example that illustrates this:

 /* DO NOT COPY THIS EXAMPLE. */

if (empty($_SESSION['authed']))
{
    header('Location: http://example.com/');
}

if (!empty($_POST['text']))
{
    /* insert into database */
}

/* Do other admin stuff */

The problem here is that the developer does not stop script execution after issuing the redirect. While the result when testing this code will be as expected (a redirect happens when the user is not logged in and tries to access the link). There is however a gaping security hole here, hidden not in what’s in the file, but what’s missing. Since the developer does not terminate the execution of the script after doing the redirect, the script will continue to run and do whatever the user asks of it. If the user submits a POST request with data (by sending the request manually), the information will be inserted into the database, regardless of wether the user is logged in or not. The end result will still be a redirect, but the program will execute all regular execution paths based on the request.