sbs20

DIY Devices - network attached storage

2021-06-05 DIY devices NAS ODroid HC2 RAID

Disk

How I went from off the shelf RAID devices to single drive Debian.

My first NAS was a QNAP TS-201. I can't remember exactly when I bought it but I think it was around 2007. It had two 1TB disks in RAID1. That was a lot of space back then - at least for me. But having lots of storage soon meant that I used it. So in 2009 I got myself a QNAP 639 Pro with an Intel CPU and slots for 6 disks. I added 3 x 3TB to start with - I couldn't justify spending more at the time and 6TB of RAID5 storage was plenty.

I soon realised I needed to be able to back everything up from there too (RAID is not backup after all). So I bought a QNAP TS 412 a few years later.

By around 2015, I was getting much more proficient with Linux and the custom firmware releases were no longer helping but hindering me. Furthermore, QNAP was no longer really supporting the older devices - there were still security patches, but no feature releases. And it was difficult to install other non-standard things on there if I wanted to. I don't blame QNAP, they have to draw a line somewhere as to what they support but it didn't help me.

Time to go DIY. #

Around this time I discovered Brian Moses and his DIY NAS blogs. Brian is typically much more interested in the hardware and build than I am. I find myself frequently having different needs or coming to different conclusions to him, but his writing was and remains both interesting and useful. If you're interested in this kind of thing I would strongly recommend his blog if for no other reason than a different perspective.

In the end a friend recommended an HP ProLiant Microserver Gen 8 to me - for some reason there was a deal on at the time which meant I could pick up a 4 bay Celeron with 4GB RAM for around £110. £110 for an Intel device on which I could run whatever distro I wanted. I had looked at ARM alternatives but not only were they more expensive but one of main requirements was to be able to run Dropbox with the native client - which requires Intel.

Proliant

Software next. My options were:

Over the course of the following couple of years I used OpenMediaVault. I managed to move the QNAP disks over without reformatting and just mounted the mdadm partition (see appendix for more). But I didn't need or want the web UI - for me it was just another vector of attack having simple password authentication. And while the plugins are great for print server / rsync and so on, I just didn't need all that many features and wanted to remain on the command line. So I switched to Ubuntu Server. But as with my router build I couldn't get on with Ubuntu and then switched again to Debian which I run today.

Aside: There's ultimately not a good reason for being CLI other than slightly lower resource usage, easier to secure, a lower risk of abandonware (Debian isn't going anywhere for a while) and most importantly: I wanted to. It started because I only wanted to access using key-based SSH - but I could still have done that with port forwarding and running the web on localhost only. Ultimately it was just a way of forcing me to become more acquainted with the command line, use only what I needed to, and also automate all the configuration I wanted so that when I needed to rebuild, I could just run a script.

Backup #

ODroid HC2

A background thought throughout this whole process (over a couple of years) was that I found RAID annoying. There was always some disk on the verge of failure. And each disk was quite expensive and became a limiting factor in the overall size of the storage array. Restriping was time consuming and it was just a headache. And ultimately, RAID is not a backup. I did some maths and found that a single 12TB disk was comfortably cheaper than 4x4TB. And for backup I didn't need awesome performance - just "good enough". So as an experiment, I purchased an ODroid HC2, put in a 12TB disk, installed Armbian and setup a cronned rsync over SSH bash script from the primary storage. I get around 600mbit/s. It's not gigabit, but it's good enough. The only noteworthy thing about the whole process was that the HC2 was fussy about which SD card to use. Other than that I now have a small, quiet, low powered backup device in a different physical location for as little money as possible (about £400 including SD and disk).

UnRAID #

Back to the primary storage (the HP Microserver). I no longer wanted to use RAID. I stopped the sharing services (CIFS/SAMBA), did a backup to the ODroid copied the data from the RAID partition to another new 12TB disk, removed the old 3TB disks, slotted in the new drive, mounted, verified the data and all done. I kept the old disks for a bit just in case, but now just have three spare slots. When I want to expand storage, I will buy a new 18TB disk (or however big I can get when I need it), put it in the ODroid, backup, and put the old disk in one of the spare slots of the HP.

Today #

I have simple scriptable NAS storage for capacities up to 12 - 18TB. Backups over SSH (very small attack surface). Mainline distro support and performance. I also upgraded the CPU in the HP Microserver. This was useful in the process - I just ordered a XEON E3 1265L v2 on eBay from Hong Kong - it took a few weeks to arrive but only cost £70. I also run CUPS for print, have my scanner connected, use it for remote development (e.g. scanservjs) and also run a second read-only SMB daemon using docker on a different NIC (on a different VLAN) for Kodi clients.

Appendix - Migrating from QNAP to Debian #

QNAP QTS is just Linux but I didn't know if the existing partitions on the QNAP would be readable on my new NAS. Short answer: yes, it can just work. Please note that I am no expert in RAID so you may find better, more reliable witnesses than me. But for what it's worth this is what I learned:

Compatibility #

  1. RAID needs a controller to mediate disk access.
  2. RAID controllers can be either hardware or software
  3. Hardware RAID controllers are expensive. If you're reading this, then you're probably more of the software RAID type.
  4. (A hardware RAID controller for the HP Proliant Servers costs ~$400)
  5. Software RAID is free and is used with less expensive, consumer grade devices. One such RAID controller is mdadm
  6. I do not know this, but I am willing to speculate that most if not all consumer NAS devices use mdadm.
  7. I do know that QNAP does and any Linux can use mdadm

QNAP partitions #

Just by way of background:

Hypothesis #

It's possible to migrate RAID disks from any proprietory NAS device (QNAP, Netgear, Synology, WD etc) to any linux host which supports mdadm provided there is an existing destination host OS.

Migrating #

So, I only did this once. Maybe I was lucky. But in any case, I'd backed everything up first. Please do the same. Or know that this is now your responsibility.

Also note that you probably won't be able to migrate from QNAP to another proprietory manufacturer e.g. Netgear, Synology etc. I expect (but do not know) that they each store their firmware in a similar way to QNAP. If QNAP doesn't recognise what's being plugged in it will destroy the data and format. I expect other NAS devices to do the same.

That said, migrating to a pre-existing Linux install, we're not doing anything too destructive so you can probably abort the process even near the end.

  1. Backup the data partition
  2. Move the disks in the correct order. i.e. Bay 1 to bay 1, 2 to 2, etc
  3. Boot the destination (e.g. Debian or OMV)
  4. Look in RAID Management on OMV, you should see your filesystem and get its device name e.g. /dev/md125. The command line alternative is cat /proc/mdstat)
  5. Go to File Systems, highlight the device (/dev/md125) and mount (CLI: sudo mount /dev/md125 /mnt/raid). At this point you should already be able to look at the data (ssh and look around). If it works then you'll want to add to fstab if you're using CLI only.
  6. To share then go to Shared Folders, add, select device, then the folder... Give it a name and save - this step is only applicable to OMV.
  7. Go to your services and SMB/CIFS then add your share. If you're on the CLI then you need to install samba and configure it).
  8. Double check that you can access your data.
  9. Reboot.
  10. If everything is ok.... and you know you do not want to go back.... then you can delete the other filesystems on the disks - i.e. the smaller (500mb) firmware partitions.