Check your backups

Just a reminder for the IT minded SF members.

If you have data which important and for which you have backups then check periodically that you a) have the files and backup files that you expect and b) that you can read them.

A hard drive died the other day - not a big deal (or so I thought), it had half a RAID array (so that’s not a problem unless the disk on which the other half resides dies** before I add a new drive into the system) and a backup copy of pretty much all the videos that we have of my son growing up, plus other sundry backup files.

No worries I thought, when the new drive arrives (which was yesterday) I’ll copy the originals from the video editing machine.

Except that my “originals” were not there and the “backups” were the only copy so I thought I’d lost just about everything.

Obviously the machines have had upgrades at various points in the past and I clearly copied the files off the other PC during the last refit, but forgot to copy them back. Oops.

As luck has it I found some disk images from even further back and, fortunately, I think all the material I thought that I’d lost is in one of them so I have probably “got away with it” ths time, or lost only a small amount (the disk is dying rather than dead so I might be able to get a bit off it before it permanently points its digital toes skywards).

So, remember
a) you can never have too many backups
b) keep copies of important data in at least three places - at least one of which is physically separate from the others
c) periodically check the health of both your original data and your backups.

** more common than you think. Or it dies during the re-silver given the extra stress of a full read of all the data in copying it to the new array member.

12 Likes

I am definitely not IT minded, but everything I do is done, not to the computer but to an external drive, and immediately copied to a 2nd one. Then once a month I copy the lot to a 3rd.

4 Likes

When I used to be involved in backups for work, we followed the 3-2-1 principle… 3 copies, on 2 different media, with 1 copy being off-site. This was back when tape storage was the best method for long-term storage. If I was doing it today, I’d probably replace transporting tapes to an off-site location with cloud storage.

For what it’s worth, I use BackBlaze for some of my personal backups, as they give you 10GB of storage for free.

3 Likes

My routine backups are done disk to disk with occasional copies taken to France; there actually should be a copy of the videos in France as well but I now realise I didn’t copy the home videos back to the machine when I had it over for repair/update (though copies were taken of the photos and archived backups from the main system) - I might have spotted issues at the time had I done so.

As I said the “gotcha” was not having duplicate copies when I thought that I did - consequently let my eye off the ball for that set of data.

Do you keep one of the copies elsewhere, even if it’s just in an outbuilding?

An insurance company client of mine used to have an HQ in the World Trade Centre 2 in New York. They had, as you’d expect, a regular backup routine with duplicate copies held off-site. In WTC 1…

1 Like

(d) (or is that a subset of (c)?) - periodically check that you can restore your data completely from the backup.

1 Like

Pardon my ignorance but doesn’t cloud storage do away with the need for additional back-ups? I assume (perhaps foolishly) that the cloud storage company will have all the ‘belts-and-braces’ back-ups and copies needed?

There’s that word again! Cloud storage companies are not infallible so it’s worth having a second (or third) copy of essential files just in case.

1 Like

TBH I wouldn’t count cloud storage as “a copy” for backup purposes at all - certainly not without an SLA with an agreed backup plan, frequency, granularity, time to recovery etc. In fact I would not be surprised to find low cost cloud storage offering no, or only “best effort” backup - plus you have the additional failure modes of “forgot to pay subscriptions” and “company went bust”.

3 Likes

A few years ago I had a disk failure - Windows was unable to ready anything.
Fortunately I had a copy of Puppy Linux and was able to read / recover most of the data using that.

So maybe worth trying to read the disk again using a different OS?

E.g. for Google Workspace - disaster recovery

I’m already a Linux user - I suspect Windows would not do a better job, especially as it is an ext4 filesystem.

Sometimes temperature can be a factor - I’ll set up a scratch system so that I can point a couple of fans at the drive and see if that helps pull any data from it.

Re-reading my post I think I should clarify - I’m thinking about stuff like OneDrive and similar. There are services aimed specifically at cloud backup solutions and if you are using one of those it is probably OK

I’d stick with my comment regarding understanding the service that is offered though.

No, it isn’t really fireproof, just that I feel so much better not having all the eggs in one basket.
BTW, for me, clouds are for bringing rain, don’t even trust them for that either. :roll_eyes: :rofl:

1 Like

I back up every 2 weeks. I have two separate external 1TB USB hard drives for the purpose and swap between them every backup. I also always do a verify after backup to make sure it’s been done properly. It takes time, but it’s worth it for the peace of mind. All my astro images are backed up on Blu-Ray disks as this is a one-shot thing.

If anyone wants to do backup I would highly recommend Macrium Reflect, which has a free version for personal use that is very similar to the paid for version and is very good. I’ve used it for many years now.
Edit: Macrium Reflect is Windows only, so Linux/iOS users would need something else.

3 Likes

Sometimes it’s the electronics that fail rather than the medium, especially if it’s an intermittent issue. If you have access to another disk of the same model, you can try swapping the boards. Most are just held in by a few torx screws. I’ve done this in the past at work with success although it probably is a long shot.
Of course, failing electronics can cause disk corruption as well, but continuing to use it if it is the elctronics will do more and more damage.

1 Like

Indeed, probably not in this case as it is “alive” and well in the sense of talking to the drive but access is producing (un)recoverable read errors so the log is full of this sort of thing

Sep 25 22:47:04 ratty kernel: ata13.00: exception Emask 0x0 SAct 0x8 SErr 0x0 action 0x0
Sep 25 22:47:04 ratty kernel: ata13.00: irq_stat 0x40000008
Sep 25 22:47:04 ratty kernel: ata13.00: failed command: READ FPDMA QUEUED
Sep 25 22:47:04 ratty kernel: ata13.00: cmd 60/08:18:68:90:44/00:00:13:03:00/40 tag 3 ncq dma 4096 in
                                                       res 43/40:08:68:90:44/00:00:13:03:00/40 Emask 0x408 (media error) <F>
Sep 25 22:47:04 ratty kernel: ata13.00: status: { DRDY SENSE ERR }
Sep 25 22:47:04 ratty kernel: ata13.00: error: { UNC }
Sep 25 22:47:04 ratty kernel: ata13.00: configured for UDMA/133
Sep 25 22:47:04 ratty kernel: sd 12:0:0:0: [sdh] tag#23 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=3s
Sep 25 22:47:04 ratty kernel: sd 12:0:0:0: [sdh] tag#23 Sense Key : Medium Error [current] 
Sep 25 22:47:04 ratty kernel: sd 12:0:0:0: [sdh] tag#23 Add. Sense: Unrecovered read error
Sep 25 22:47:04 ratty kernel: sd 12:0:0:0: [sdh] tag#23 CDB: Read(16) 88 00 00 00 00 03 13 44 90 50 00 00 00 08 00 00
Sep 25 22:47:04 ratty kernel: critical medium error, dev sdh, sector 13208162384 op 0x0:(READ) flags 0x800 phys_seg 1 prio c>
Sep 25 22:47:04 ratty kernel: ata13: EH complete
Sep 25 22:47:04 ratty kernel: md/raid1:md121: read error corrected (8 sectors at 323258448 on sdh2)

So I suspect that the electronics themselves are OK.

I reckon the rats have eaten your kernels :thinking:

3 Likes

Possibly though ratty was actually a water vole.

6.4 Terra bytes ?
Sorry. Slipped 1024. Only 600 odd GB I think. Assuming 512 byte sectors.

Edit: I was wrong. 13208162384 512 byte sectors is 6.44 TB. 4K sectors would make it 51.6 TB. Is your hard drive that big ?

1 Like