Getting my head around a new backup system

 

Warning .... this is just a brain dump really while I wade through the crap that is BackBlaze's confusing product + thirdparty software you need, the aftermath of crashplan and trying to narrow down what it is I need backing up where.

Crash plan is dead.... thank God for that. Despite the software being very good, very useful, sensibly thought out for normal people who have normal household backup needs, the cloud service sucked. The way they exited the consumer market too, it just shows how cloud services in general should not be 100% relied upon. ALWAYS HAVE A CLOUD EXIT STRATEGY.

Anyway, in the midst of everyone moaning about Cloudflare, BackBlaze, the only other real cost effective competitor, went on a gloating spree to say how good they are happy to accept new customers. So let's dive into my backup needs, what is available and what it costs.

My Backup responsibilities

Machines to backup (some needless detail too cos I'm a geek);

  • Laptop (My Lenovo X250 180GB SSD) dual boot Win10 (100GB) and Linux (80GB)
  • Laptop (Girlfriends MacBook Pro 80GB HDD) MacOS
  • Laptop (Girlfriends Lenovo T450 180GB SSD) Win10
  • Desktop (AMD A10 6800k 250GB SSD) Win10
  • NAS (HP N40L Microserver w/12TB Storage) Ubuntu LTS

Requirement

I want program that I can install on each machine/partition which backs up to both my NAS and a cloud service.

  • If I don't backup my girlfriends machines, they won't get backed up, so it needs to be done without her intervention
  • It shouldn't have to cost more than the family backup plan with CrashPlan ($150 per year total)

Previous Setup

We had a CrashPlan family plan and was installed on all the laptops (and partitions) and Desktop computer. CrashPlan was also installed in headless mode on the NAS. So all of the laptops and desktop backed up to both CrashPlan Cloud and my NAS.

The NAS itself has a more powerful twin (A N54L at my Dads house). The NAS has about 500GB of important data which I was backing up to the CrashPlan cloud (photos, emails, documents, previous work that kind of stuff) and the rest is stuff that I can re-download, re-encode or reprocess (sure it'd be a pain but it's not the end of the world, hence why I have the N54L twin).

So in all I have three operating systems, six instances that will have around 1TB max of data total to backup to the cloud, five instances (the laptops and desktop) which will back up to my NAS for local backup. 

The BackBlaze Offering

BackBlaze personal is $5 per month per PC or Mac for Unlimited backup.

Clearly this is very strict and they are trying to protect themselves against going the way of Crashplan. I understand this.

However for the following reasons, BackBlaze personal isn't an option;

  • To backup only the Windows and Mac instances, it'd be $20 per month to back up
  • I want to be able to sort all the backups in one account (like a family plan!)
  • I'd need a secondary piece of software on each machine to backup to my NAS
  • I'd need another solution Linux Laptop partition and however I deal with the important data on my NAS

To use BackBlaze with Linux use B2

Okay, so this isn't an unlimited plan, but since it's less that 1TB I'm trying to backup here then the cost shouldn't be so bad and I really do understand why BackBlaze are being an arse about this. However, here are the points here;

  • Simple pricing
    • Storage - $0.005/GB/Month
    • Download - $0.02/GB
    • Class "A" transactions – Free
    • Class "B" transactions - $0.004 per 10,000 with 2,500 free per day.
    • Class "C" transactions - $0.004 per 1,000 with 2,500 free per day.

Simple right, oh, here is the simple API Calls chart with all the transaction limits and what the transaction actually is that you need to be aware of https://www.backblaze.com/b2/b2-transactions-price.html .... Just as an aside, as much as I enjoy IT work, SysAdmin etc. when I get home I don't really want to have to do the same cloud costing bulls&*t that I have had to do at work.

So using the B2 Cost Calculator it looks like backing up 500GB with a monthly churn of say 30GB and no downloads (I assume this means recovery and not some sort of sync cost?) then it's looking at around $40 per year. This looks really reasonable, however in the estimator it doesn't not mention anything about "transactions", let's assume that I won't go over the daily limit, because nothing ever goes wrong when you assume you understand a pricing plan right? 

Pricing aside, how do you actually backup to B2?

Well, you need some software from somewhere else as BackBlaze only provide the storage and API to this. 

This is where I get quite annoyed with BackBlaze claiming to be a viable alternative to Crashplan. B2 is simply NOT a set once and forget backup solution. This is clearly designed as a cloud storage solution to rival S3 and in that it succeeds. But as an actual backup solution, this is 100% dependent on a now 3rd party in this backup solution.

I suppose we should look at some software

  • ArqBackup - $49 one-time fee per machine user (+$29.99 per licence for lifetime upgrade), Mac and Win only.
    • It can backup locally to SFTP so that's a plus
    • Looks very nice, well supported and easy to use
    • Two licences will cost me $196 $99.98 + ($59.98 for lifetime upgrades) + the backup cost
      • CORRECTION 11-09-2017 Licence improvement URL
      • Micheal Bennett kindly contacted me to let me know that ArqBackup now provide the licence per user rather than per machine. This greatly decreases this cost.
    • I still need a solution for the Linux part
  • duplicati duplicati.com - Free! - Linux, Mac, Win
    • This looks at the moment to be my best bet
    • Still in Beta, I've tried the Linux version and it seems okay for NAS backup so far
    • B2 support is quite new
    • If the Windows/Mac version is okay I may do this and donate the difference between the Crashplan family plan and the BackBlaze storage to the Duplicati developers
  • CyberDuck
    • Not a backup utility, it's a file transfer program like WinSCP
    • As a file browser though, this is awesome, just not for my needs today.
  • GoodSync $29.99 per computer
    • Quite cheap, no yearly cost
    • Might be okay for me but certainly not automated and invisible enough for my girlfriends needs
    • If I'm going to use something as complex as this, I may as well use duplicati
  • qBackup $29.99 per computer - $99.99 for a linux licence!!! WTF!?!
    • After seeing the Linux price, I stopped downloading. What a cheek!
    • So from the manual, it looks similar to GoodSync
    • The website doesn't look like it's been updated in a while, I don't trust it.
  • Cloudberry - $29.99 per computer
    • AGAIN! WTF!? They limit you to 1TB storage.... but they don't offer storage. What kind of crap is this!?
    • Sure I'm not using 1TB but why should the 3rd party vendor dictate how much I store when they're not even storing it?!
  • duplicacy.com - $20 per computer + $5 per year per computer
    • Not as much as Arq but adds a yearly cost per machine
    • Doesn't look as feature rich as Arq, but that could just be the lack of info on the website
      • I must admit that if I'm looking to pay for a backup solution, first appearances of the website really seem to matter to me
    • I may try this out but probably only if duplicati isn't suitable which I suspect won't be the case

 

Other sensible solutions?

Office 365?

This one isn't quite for me, but it maybe in the future. I'm an academic so I get a work copy of MS Office via 365. However my girlfriend doesn't have this. So, for £59.99 per year (£5 per month), you can get a personal Office 365 account giving you all the products as well as 1TB oneDrive storage, way more than needed.

This can be further extended to £79.99 per year for a family version and you can have up to 5 users each with 1TB storage on oneDrive.

This could be coupled with any of the same pieces of software you have to have for B2.

 

Conclusions?

I will finish this when I have some conclusions. In the mean time, I hate the cloud!

undefined

Backing up to AWS Glacier

 

I wanted a simple solution to backing up my webserver and the low cost of AWS S3 and Glacier is quite appealing. The solution I settled on was to use s3cmd (http://s3tools.org/s3cmd). After setting up a api key and then running;

s3cmd --configure

filling the information in. 

I created a new sql user that could read all tables for the backup and the backup user also has read access to the /var/www directory. Then this script runs nightly;

#!/bin/sh

DATE=`date +%Y-%m-%d`

#make db backup and compress
mysqldump -uuser-BUP -pPASSWORD --all-databases --result-file=/home/user-BUP/database_BUP/all_databases_$DATE.sql
gzip /home/user-BUP/database_BUP/all_databases_$DATE.sql

#transfer to S3
s3cmd put --storage-class=STANDARD_IA /home/user-BUP/database_BUP/all_databases_$DATE.sql.gz s3://serverbackup

#remove db dump as we will have loads of them
rm /home/user-BUP/database_BUP/all_databases_$DATE.sql.gz

#compress websites
tar cfzv /home/user-BUP/database_BUP/all_websites_$DATE.tar.gz /var/www/html

#transfer websites to S3
s3cmd put --storage-class=STANDARD_IA /home/user-BUP/database_BUP/all_websites_$DATE.tar.gz s3://serverbackup

#remove website compress too as we will have loads of them and these will be large
rm /home/user-BUP/database_BUP/all_websites_$DATE.tar.gz

So now that we can transfer data to S3, I added this as a cronjob;

0 00 * * * /home/mjv08/database_BUP/mysql_dump_script.sh > /home/mjv08/database_BUP/db_backup.log

With the data now backing up nightly to S3 I set up a LifeCycle rule to automagically transfer S3 data to Glacier after 14 days.

undefined

 

Finally after a few days (well after 14 days), a quick check to ensure that the transition worked and all looked great. 

undefined

Home