Users browsing this thread: 1 Guest(s)
z3bra
Grey Hair Nixers
So quick update, my HDD definitely died, and I gave up on it… It's still plugged onto my computer, "just in case", but I doubt I'll ever retrieve my data.

In reaction to this, I finally automated my backup system, which involves the following:
  • safe - to store API tokens, private keys, passwords, …
  • dedup - encrypted + compressed deduplicating snapshoting system
  • drist - automated configuration (similar to ansible/terraform)
  • rclone - rsync for "online clouds" (used for backblaze)
  • tarsnap - backup my servers' configuration (automated via cron)
  • backblaze - cheapest online storage provider (B2 storage)

All backups are done from within "cron", and I'll receive an email if any command produce an output (basically, meaning an error).

I configure my servers using drist. It also configure the backup system, by pushing the tarsnap private key on each server, as well as the list of directories/files to back up. I try to only backup what I really need, so I don't have to skim through a huge tarball and guess which file I need or not in case of recover.
These backups are done every week from within /etc/weekly.local, so I get a report whenever they run. I trust tarsnap on the topic of encryption, so I don't bother encrypting the data before sending it to their servers.

For my personal computer, I use backblaze storage, which is not encrypted. Tarsnap can be expensive when you have lots of data, so I decided on keeping it ONLY for config files (a few MB at most).
On backblaze I have 3 buckets: one for my password store, one for the drist configs, and the last one for my data (pics, videos, …).

As I used safe, all my passwords are stored encrypted on disk, so I just upload them as-is. For drist, I want to store it encrypted, and possibly version it. The best way to do this is using dedup, which basically deduplicate data, and store "snapshot" archives after encrypting and compressing it. The dedup key used to encrypt the repo is obviously stored in the safe I mentioned earlier :)
Then, I push the whole dedup repo on backblaze using rclone, knowing that my data is safe. I do the same for my other data (dedup + backblaze).

All I need now is a way to practice recovery !


Messages In This Thread
Backing up and Deploying - by venam - 29-05-2016, 12:01 PM
RE: Backing up and Deploying - by z3bra - 07-09-2016, 09:14 AM
RE: Backing up and Deploying - by TheAnachron - 07-09-2016, 09:37 AM
RE: Backing up and Deploying - by z3bra - 07-09-2016, 10:45 AM
RE: Backing up and Deploying - by TheAnachron - 09-09-2016, 04:20 AM
RE: Backing up and Deploying - by vain - 25-09-2016, 12:55 PM
RE: Backing up and Deploying - by venam - 25-09-2016, 01:33 PM
RE: Backing up and Deploying - by vain - 25-09-2016, 03:19 PM
RE: Backing up and Deploying - by josuah - 02-01-2017, 04:52 PM
RE: Backing up and Deploying - by pranomostro - 03-01-2017, 09:00 AM
RE: Backing up and Deploying - by jkl - 03-01-2017, 10:39 AM
RE: Backing up and Deploying - by venam - 03-01-2017, 11:09 AM
RE: Backing up and Deploying - by z3bra - 31-07-2020, 03:55 PM
RE: Backing up and Deploying - by opFez - 31-07-2020, 07:44 PM
RE: Backing up and Deploying - by z3bra - 11-08-2020, 04:45 AM
RE: Backing up and Deploying - by swathe - 30-05-2016, 01:44 AM
Your backup solution? - by TheAnachron - 07-09-2016, 05:14 AM
RE: Your backup solution? - by venam - 07-09-2016, 05:21 AM
RE: Your backup solution? - by TheAnachron - 07-09-2016, 05:58 AM
Backups! - by z3bra - 19-04-2019, 08:39 AM
RE: Backups! - by venam - 19-04-2019, 10:00 AM