Tech Stuff

Home Lab

Cisco Tips & Tricks

FreeNAS

Ham Packet Radio

Pocket Stocks

 IPv6 Torrent Tracker

Personal

Photos

Video

Metal Casting @ Home

Old Stuff

WM Smartphone Bittorrent

New Page 1

FreeNAS - Open Source Storage Server

I have been looking for a very cheap mass storage solution for home and test environments.  My needs are:

  • Redundancy - Any one drive can fail without data loss

  • Filesystem expansion - Disks can be added to expand the existing filesystem

  • CIFS/SMB sharing - For Windows clients

  • iSCSI target - To host ESX datastores

  • Easy administration with command line interface for troubleshooting

I settled on Sun's ZFS as the filesystem, since it offers both expansion and redundancy.  ZFS is native to OpenSolaris, but I decided on FreeNAS 0.7---a Linux distro aimed at running cheap x86-based storage servers.  It provides a relatively simple interface to share using CIFS/SMB, iSCSI, FTP, SSH, etc.  I've discovered some neat things while reading blog posts and testing on a virtual machine.  In the examples below, I list the CLI commands because the FreeNAS GUI doesn't automate some of the more advanced steps.

1) Create a basic ZFS pool
2) Add space to an existing ZFS pool
3) Expand an existing vdev

 

Create a Basic ZFS Pool

Creating a ZFS pool from the command line is very simple:

zpool create tank mirror da0 da1

(where "tank" is the name of the ZFS pool, and we're creating a virtual device made up of two mirrored disks:  da0 and da1)

When that command is executed, ZFS automatically creates the filesystem and mounts it as /tank.  If you create it from the FreeNAS GUI, it will be mounted at /mnt/tank.  I created a pool from two 2GB disks on VirtualBox:

freenas:~# zpool status tank
pool: tank
state: ONLINE
scrub: none requested
config:

NAME            STATE       READ    WRITE      CKSUM
tank                ONLINE           0              0                 0
    mirror         ONLINE          0               0                 0
        da2         ONLINE          0               0                 0
        da3         ONLINE          0               0                 0

errors: No known data errors


freenas:~# zpool iostat tank
                        capacity            operations        bandwidth
pool             used      avail       read   write       read   write
----------        -----       -----        -----     -----        -----     -----
tank           116K     1.99G          0         0           19     208

Take a minute to understand the ZFS hierarchy:  A zpool is essentially a filesystem.  It is made from virtual devices (vdevs).  A vdev can be made from a single disk, a mirrored set of disks, or a RAID set of 3 or more disks.  ZFS tries to use all space on the vdevs without considering data redundancy.  It assumes the vdevs presented to it are failure resistant.  If you're creating a zpool for a fileserver, you want to add disks as a mirrored vdev or RAID-Z vdev.

 

Add Space to an Existing ZFS Pool (while online!)

The ZFS pool and filesystem can be expanded without taking it offline.  Let's add a mirrored vdev to the above pool using the new 1GB da0 and da1 drives:

freenas:~# zpool add tank mirror da0 da1
freenas:~# zpool status tank
pool: tank
state: ONLINE
scrub: none requested
config:

NAME            STATE       READ    WRITE      CKSUM
tank                ONLINE           0              0                 0
    mirror         ONLINE          0               0                 0
        da2         ONLINE          0               0                 0
        da3         ONLINE          0               0                 0
    mirror         ONLINE          0              0                 0
        da0          ONLINE         0               0                0
        da1         ONLINE          0               0                0

errors: No known data errors

And verify that the pool size has increased:

freenas:~# zpool iostat tank
                        capacity            operations        bandwidth
pool             used      avail       read   write       read   write
----------        -----       -----        -----     -----        -----     -----
tank           124K     2.98G          0         0           16     414

Very cool.  If you're using SATA, SCSI, or USB drives, you can use this method to add storage without ever taking anything offline.

 

Expand an Existing VDEV

I started my ZFS/FreeNAS server using a 10GB boot drive and a mirrored pair of 40GB drives for the zpool.  If I continue to expand using internal 1TB-2TB drives, I will eventually want to get rid of those 40GB drives to save space.  Unfortunately, ZFS does not allow a vdev to be removed.  So I'm stuck with that 40GB mirror vdev until I destroy the zpool (and data!).  Fortunately there's a workaround.  We can replace each 40GB drive with a 1TB drive, and allow the system to resilver (rebuild) onto the new drives.  This must be done one drive at a time to avoid data loss.  Once that is done, we need to unmount and remount the filesystem.  I have seen a sample where an unmount/remount was not needed, so perhaps it can all be done online using the newest ZFS release.  Here's the example on FreeNAS, using 1GB and 2GB drives:

freenas:~# zpool create tank mirror da0 da1
freenas:~# zpool status tank
pool: tank
state: ONLINE
scrub: none requested
config:

NAME          STATE        READ    WRITE    CKSUM
tank              ONLINE             0             0                0
    mirror       ONLINE             0             0               0
        da0       ONLINE             0             0               0
        da1       ONLINE             0             0               0

errors: No known data errors

freenas:~# zpool iostat tank
                     capacity      operations      bandwidth
pool        used       avail    read   write    read   write
----------   -----        -----      -----    -----     -----     -----
tank       111K    1016M        0       10      78    23.8K

freenas:~# zpool replace tank da0 da2
freenas:~# zpool replace tank da1 da3
freenas:~# zpool status
pool: tank
state: ONLINE
scrub: resilver completed with 0 errors on Mon Jan 18 00:57:41 2010
config:

NAME          STATE        READ    WRITE    CKSUM
tank             ONLINE             0              0               0
    mirror      ONLINE             0              0               0
        da2      ONLINE             0              0               0
        da3      ONLINE             0              0               0

errors: No known data errors

freenas:~# zpool iostat tank
                     capacity          operations      bandwidth
pool         used       avail     read   write     read   write
----------    -----        -----      -----     -----       -----    -----
tank        218K    1016M        0         4       358   9.49K

ZFS doesn't recognize the extra capacity, even though we've swapped up to 2GB disks.  We need to export and import the zpool, at least when using FreeNAS.

freenas:~# zpool export tank
freenas:~# zpool import tank
freenas:~# zpool iostat tank
                     capacity         operations      bandwidth
pool          used     avail     read   write     read   write
----------     -----      -----      -----     -----       -----     -----
tank        116K    1.99G        5        29    6.54K   71.7K

All set!  Too bad this isn't an online procedure with FreeNAS.

Your IP address is: 38.107.191.80