FISH
I do a good bit of hardware integration with the web, with manufacturing equipment, with embedded systems and with big data set, or that can sustain multiple failures. Not necessarily all at once, but typically, people expect FISH from me :)
FISH is Fully Integrated Software and Hardware (btw, as a side note, the internal project at Sun to create appliances based on ZFS was known as FISHWorks). The Raspberry Pi is a cool piece of hardware, but I typically need stuff that is only (or mostly) found on Solaris and derived OSes, such as ZFS. I've been using ZFS for many years now, since the first public release on Solaris Nevada. ZFS scales and give you data integrity. And it can run on the largest systems known to man.
It scales
For example, I'm listening right now to
ZFS Day's live video stream and hearing a talk about ZFS on the
Sequoia supercomputer, which is the fastest supercomputer out there. They are using it as a native port, not using
FUSE.
What is ZFS?
Wikipedia: "ZFS is a combined file system and logical volume manager designed by Sun Microsystems. The features of ZFS include data integrity verification against data corruption modes, support for high storage capacities, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z and native NFSv4 ACLs. ZFS is implemented as open-source software, licensed under the Common Development and Distribution License (CDDL)."
From Supercomputers to $35 computers
So, ZFS scales at the highest level obviously. Well, it also scales down: I've been using a bit ZFS on the Raspberry Pi using FUSE, until I can get a
Solaris derived OS ( such as illumos, smartos, openindiana, opensolaris etc) on the Raspberry Pi. That way, at least I have ZFS. Still missing zones, smf and dtrace, but it is a start.
Now just a reminder, the Pi only has 256MB total ram, and a BCM arm processor. So first thing first, we need to give as much ram to the OS as possible, and reduce the video buffer size:
I'm using a 240MB split on that Raspberry Pi since it is running only in text mode at the console, and I remote to it using ssh -X.
If you use the composite out you might want to use the 224MB split and definitely 192 or 128 using HDMI, but then at that point, you are chocking ZFS. That's 128 for OS and ZFS and whatever apps you are running...
Fully loaded
Altough
Raspbian comes with a good amount of stuff preloaded, it was not intended to be used with FUSE out of the box, and ZFS was probably never on the radar screen of anybody. So let's start with adding the FUSE stuff and the libraries and tools we will need to build ZFS. This is the shortlist:
fdion@raspberrypi ~/zfs $ sudo apt-get install fuse-utils libfuse-dev libfuse2
fdion@raspberrypi ~/zfs $ sudo apt-get install libaio-dev libattr1-dev attr
fdion@raspberrypi ~/zfs $ sudo apt-get install git scons
If you build it...
So we have the prerequisites. Let's get the code, compile it and install the tools:
fdion@raspberrypi ~ $ mkdir zfs
fdion@raspberrypi ~ $ cd zfs
fdion@raspberrypi ~/zfs $ git clone https://bitbucket.org/cli/zfs-fuse-arm.git
fdion@raspberrypi ~/zfs $ cd zfs-fuse-arm/
fdion@raspberrypi ~/zfs/zfs-fuse-arm $ cd src
fdion@raspberrypi ~/zfs/zfs-fuse-arm/src $ scons
[a lot of stuff will scroll by]
fdion@raspberrypi ~/zfs/zfs-fuse-arm/src $ sudo scons install
[again, more stuff will scroll by]
Wow, it compiled (scons). And installed (sudo scons install). It's a good thing we are using the zfs-fuse-arm version, because the mainline wont go very far on the compile.
A demonstration, if you please?
Well of course! Let's start the zfs-fuse daemon and create two virtual disks. I'm creating two 100M disks here using dd/ (this is on a slow SD card, rated 10MB/s). You could also use an actual /dev (like a pair of USB keys):
fdion@raspberrypi ~/zfs/zfs-fuse-arm/src/zfs-fuse $ sudo sh run.sh &
fdion@raspberrypi ~/zfs/zfs-fuse-arm/src/zfs-fuse $ cd
fdion@raspberrypi ~ $ cd zfs
fdion@raspberrypi ~/zfs $ mkdir test
fdion@raspberrypi ~/zfs $ cd test
fdion@raspberrypi ~/zfs/test $ dd if=/dev/zero of=fakedisk1 bs=1024k count=100
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.2747 s, 10.2 MB/s
fdion@raspberrypi ~/zfs/test $ dd if=/dev/zero of=fakedisk2 bs=1024k count=100
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.7517 s, 9.8 MB/s
Up to now we haven't done anything with ZFS per say. And basically to mirror two drives in ZFS and create a new storage out of that, all we have to do:
fdion@raspberrypi ~/zfs/test $ sudo zpool create mymirror mirror /home/fdion/zfs/test/fakedisk1 /home/fdion/zfs/test/fakedisk2
Now let's create a filesystem on that new zpool device, and mount it to a local folder in my home directory, change permissions so I can write to it and finally copy some files from /etc to my new filesystem:
fdion@raspberrypi ~/zfs/test $ cd
fdion@raspberrypi ~ $ mkdir myfilesystem
fdion@raspberrypi ~ $ sudo zfs create mymirror/myfilesystem -o mountpoint=/home/fdion/myfilesystem
fdion@raspberrypi ~ $ sudo chown fdion:pi myfilesystem/
fdion@raspberrypi ~/myfilesystem $ cp /etc/*.conf .
cp: cannot open `/etc/fuse.conf' for reading: Permission denied
fdion@raspberrypi ~/myfilesystem $ ls
adduser.conf gssapi_mech.conf libaudit.conf pnm2ppa.conf
asound.conf hdparm.conf logrotate.conf resolv.conf
ca-certificates.conf host.conf mke2fs.conf rsyslog.conf
colord.conf idmapd.conf mtools.conf sensors3.conf
debconf.conf insserv.conf nsswitch.conf sysctl.conf
deluser.conf ld.so.conf ntp.conf ts.conf
gai.conf libao.conf pam.conf ucf.conf
fdion@raspberrypi ~/myfilesystem $ sudo zfs list
NAME USED AVAIL REFER MOUNTPOINT
mymirror 191K 63.3M 22K /mymirror
mymirror/myfilesystem 89.5K 63.3M 89.5K /home/fdion/myfilesystem
fdion@raspberrypi ~/myfilesystem $ sudo zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
mymirror 95.5M 196K 95.3M 0% 1.00x ONLINE -
fdion@raspberrypi ~/myfilesystem $
How cool is that? I now have a mirrored backup of my .conf files. Well, not quite. We are using fake disks, so if the SD card dies I loose all.
So next time we'll demo with actual USB drives.
