DailyStuff

In a previous posting I started with BtrFS and as mentioned BtrFS supports snapshotting. With this you can create a point in time copy of a subvolume and even create a clone that can be used as a new working subvolume. To start we first need the BtrFS volume which can and must always be identified as subvolid 0. This as the default volume to be mounted can be altered to a subvolume instead of the real root of a BtrFS volume. We start with updating /etc/fstab so we can mount the BtrFS volume.

LABEL=datavol	/home	btrfs	defaults,subvol=home	0	0
LABEL=datavol	/media/btrfs-datavol	btrfs	defaults,noauto,subvolid=0	0	0

As /media is a temporary file system, meaning it is being recreated with every reboot, we need to create a mountpoint for the BtrFS volume before mounting. After that we create two read-only snapshots with a small delay in between. As there is currently no naming guide for how to call snapshots, I adopted the ZFS naming schema with the @-sign as separator between the subvolume name and timestamp.

$ sudo mkdir -m 0755 /media/btrfs-datavol
$ sudo mount /media/btrfs-datavol
$ cd /media/btrfs-datavol
$ sudo btrfs subvolume snapshot -r home home\@`date "+%Y%M%d-%H%m%S-%Z"`
Create a readonly snapshot of 'home' in './home@20124721-080109-CET
...
$ sudo btrfs subvolume snapshot -r home home\@`date "+%Y%M%d-%H%m%S-%Z"`
Create a readonly snapshot of 'home' in './home@20124721-080131-CET'
$ ls -l
totaal 0
drwxr-xr-x 1 root root 52 nov 21  2010 home
drwxr-xr-x 1 root root 52 nov 21  2010 home@20124721-080109-CET
drwxr-xr-x 1 root root 52 nov 21  2010 home@20124721-080131-CET

We now have two read-only snapshots and lets test to see if they are real read-only subvolumes. The creation a new file shouldn’t be possible.

$sudo touch home@20124721-080109-CET/test.txt
touch: cannot touch `home@20124721-080109-CET/test.txt': Read-only file system

Creating snapshots is fun and handy for migrations or as on disk backup solution, but they do consume space as the delta’s between snapshots is being kept on disk. Meaning that changes between the snapshots are being keept on disk even when you remove them. Freeing diskspace will not only be removing them from the current snapshot, but also removing previous snapshots that include the removed data.

$ sudo btrfs subvolume delete home@20124721-080109-CET
Delete subvolume '/media/btrfs-datavol/home@20124721-080109-CET'
$ ls -l 
totaal 0
drwxr-xr-x 1 root root 52 nov 21  2010 home
drwxr-xr-x 1 root root 52 nov 21  2010 home@20124721-080131-CET

As last step we unmount the BtrFS volume again. This is where ZFS and BtrFS differ too much for my taste. To create and access snapshots on ZFS the zpool doesn’t needs to be mounted, but then again with the first few release of ZFS the zpool needed to mounted as well. So there is still hope as BtrFS is still under development.

$ sudo umount /media/btrfs-datavol

Seeing what is possible with BtrFS, Sun’s TimeSlider becomes an option. Also the option of Live Upgrades with rollbacks as is possible with Solaris 11, but for that BtrFS with read-write snapshots needs to be tested in the near future.

After using ZFS on Solaris, I missed the ZFS features on Linux and with no chance of ZFS coming to Linux I had to do with MD and LVM. Or at least until BtrFS became mature enough and since the Linux 3.0 that time slowly has come. With Linux 3.0 BtrFS supports autodefragmentation and scrubbing of volumes. The second is maybe the most important feature of both ZFS and BtrFS as it can be used to actively scan data on disk for errors.

The first tests with BtrFS where in a virtual machine already a longtime ago, but the userland tools where still in development. Now the command btrfs follows the path set by Sun Microsystems and basically combines the commands zfs and zpool for ZFS. But nothing compares to a test in the real world and so I broke a mirror and created a BtrFS volume with the name datavol:

$ sudo mkfs.btrfs -L 'datavol' /dev/sdb2

Now we can mount the volume and create a subvolume on it which we are going to be using as our new home volume for users homedirectories.

$ sudo mount /dev/sdb2 /mnt
$ sudo btrfs subvolume create /mnt/home
$ sudo umount /dev/sdb2

When updating /etc/fstab we can tell mount to use the volumename instead of a physical path to a device or some obscure UUID number. Also you can tell which subvolume you want to mount.

LABEL=datavol	/home	btrfs	defaults,subvol=home	0	0

After unmounting and disabling the original volume for /home we can mount everything and copy all the data with rsync for example to see how BtrFS is working in the real world.

$ sudo mount -a

As hinted before scrubbing is important as you can verify that all your data and metadata on disk is still correct. You can do a read-write test by default or only read test to see if all data can be accessed. There is even an option to read parts of the volume that are still unused. In the example below the subvolume for /home is being scrubbed and with success.

$ sudo btrfs scrub status /home
scrub status for afed6685-315d-4c4d-bac2-865388b28fd2
	scrub started at Sat Jan 17 15:11:58 2012, running for 106 seconds
	total bytes scrubbed: 5.77GB with 0 errors
...
$ sudo btrfs scrub status /mnt
scrub status for afed6685-315d-4c4d-bac2-865388b28fd2
	scrub started at Sat Jan 17 15:11:58 2012 and finished after 11125 seconds
	total bytes scrubbed: 792.82GB with 0 errors

The first glances of BtrFS in the real world are a lot better with kernel 3.1 then somewhere with kernel 2.6.30 and I’m slowly starting to say it becomes ready to be included in RHEL 7 of Debian 8 for example as default storage solution. The same as ZFS became in Solaris 11. But it is not all glory as still a lot of work needs to be done.

The first is encryption as the LUKS era ends with BtrFS as it is not smart to put it between your disks and BtrFS. You lose the advantage of balancing data between disks when you do mirroring for example. But then again LVM has the same issue where you then also first need to setup software raid with MD with LUKS on top of it and LVM on top of that. For home directories EncFS maybe an option, but it still leaves a lot of area’s uncovered that would be covered by LUKS out of the box.

The second issue is the integration of BtrFS in distributions and the handling of snapshots. As for now you first need to mount the volume before you can make a snapshot of a subvolume. The same for access a snapshot and for that I think ZFS still has an advantage with the .zfs directory accessible for everyone who has access to the filesystem. But time will tell and for now the first tests look great.

Last December 10th I took the CISSP exam in Brussels and yesterday after only four weeks I received the following in my mailbox:

Dear Hans Spaans:

Congratulations! We are pleased to inform you that you have passed the Certified Information Systems Security Professional (CISSP®) examination – the first step in becoming certified as a CISSP.

So I now only need to submit my resume and endorsement. Ow and order some cake for co-workers. And the reason I did it in Brussels instead of Utrecht? I was a little bit late with requesting the exam as I did that on November 23th. Now it is time to plan the next exam, but it won’t be CEH.

I stumbled on a phishing site for a Dutch-bank in my junk-folder and for once I decided to have closer look to see if the filter was working correctly. Is was, but after reviewing the phishing site I saw two things and it was time to act.

The first one was the hosting service. It was a free hosting service so no defacing or whatever. That makes live very convenient for hosting a phishing site that looks pretty safe. The seconds was the use of a free hosting service for submit and collect forms. The funny part is btw, that the seconds appears to very if a certain tag is in the referral page, but doesn’t check if it really shows up. So to eliminate the inclusion in the webpage, the have added then after the closing HTML-tag. Maybe using XPath was a better design choice over just search for a certain string to enable the service.

As the form was asking for all kind of funny details to do perfect phishing I decide to report this to all involved parties. The site being phished, Rabobank in this case, the hoster T15.org and Formbuddy for processing phishing data. After so checking and didn’t found enough leads on alternative mail-addresses to report this I decide to use RFC 2142 reserved mail-addresses and the following happend.

<abuse@rabobank.nl>: host mail01.rabobank.nl[145.72.107.42] said: 550 #5.1.0
Address rejected. (in reply to RCPT TO command)

<security@rabobank.nl>: host mail01.rabobank.nl[145.72.107.42] said: 550 #5.1.0
Address rejected. (in reply to RCPT TO command)

<security@formbuddy.com>: host ASPMX.L.GOOGLE.com[74.125.79.27] said: 550-5.1.1
The email account that you tried to reach does not exist. Please try
550-5.1.1 double-checking the recipient’s email address for typos or
550-5.1.1 unnecessary spaces. Learn more at
550 5.1.1 http://mail.google.com/support/bin/answer.py?answer=6596
d15si7088885eei.16 (in reply to RCPT TO command)

The one that worries me the most is that a bank appears to have no working mail-addresses as described in Section 4 of RFC 2142. Those are basically key for contacting parties in case of emergencies or trouble. The abuse-reject was already noticed by someone last year, but I really wonder how a /16 network can ignore this. Also since there is no abuse-c entry know for there /16.

Update 2012-01-06: The nice guys at T15.org have taken the website down within a few hours after reporting.

2011 has been a strange year for me personally, maybe also a reason why I didn’t blog that much and hopefully 2012 will be better. But a lot has happend in 2011 as Debian 6.0 was released, GNOME 3 impacted the world a lot and still does, Linus released Linux 3.0.0 and many other things happend in the FOSS world. Also two titans, Dennis Ritchie and Steve Jobs, past away.

Also other things changed last year and one of the biggest driver behind this was my Android phone. Google looks nice in many ways and promises “not to be evil”, but I don’t trust them. It was a driver for me to set up my own CalDAV and CardDAV server and it looks fine for now, but also looking into TT-RSS as replacement Google Reader. Some things still need some love and sweet, and we will see this progresses in 2012 hopefully.

2011 was also a reasonable year for reading. The list includes “Cloud Application Architectures”, “Cloud Security and Privacy”, “Being Geek”, “Myths of Innovation” and “Network Warrior”, but also some books for CISSP. Luckily the list also includes non tech-books with a few books from “The Wheel of Time”, “Discworld”, “Ghost in the Wires” and “Steve Jobs in His Own Words”.

Like I said, 2011 was the year the cloud came into my life. And to be honest, the cloud meaning in this context, the separation of data from an application and from a local installed application. With this came also my love and hate relationship with Tor as it may be an answer to certain flaws in DNS for example where a government can take over a domain name or disable it. With this once digital life basically ends. The name resolution within Tor really looks promising as also for Tor-chat, but it is slow as hell. For chatting it could be usable, but not for browsing at the moment. But I still wonder why projects like GNU, Gutenberg or Wikipedia have no known presence on the Tor-network.

The cloud thingy made me slowly also wonder about my next workstation. I bought this machine begin 2009 and I expect to buy a new one at the end of 2012 or in 2013. Most likely it will be a laptop then, but which one? One thing I hope before then and that is that Tux goes on a diet as my root-volume currently is at 12G and I’m sure it was between the 6 a 7G a year ago. I hope it is some additional fat from running Debian Testing, but I expect not.

Also this year I finished the conversion of my music collection to FLAC to make a copy of it in Ogg Vorbis. Yes, FLAC became my archiving format and Ogg Vorbis my day to day format to make it more useful so I can also put them on my phone without filling up the 32G SD-drive with just a few CD’s. Also good and bad news for the movie-industry. Yes, I’m going to the pictures again, but only from the money I get from selling my DVD-collection. And about downloading things, that has slowed down also and my backlog slowly starts to dry up without adding anything new. The round silver disc’s are going the same way as paper in my house. Slowly almost becoming extinct.

A few things I promised myself to do in 2011 I didn’t do sadly enough. Taking up C-programming again and learn how to create decent Debian packages and related infrastructure. Hopefully I can spent some time on this in 2012, but for now I took up Latex again and I like it. About other things we will see, but looking back it was a good year where I switched from being an Unix-engineer towards a security officer. I can only hope the trend progresses, but we will see in 12 months time.