Wednesday, February 29, 2012

ZFS Cheatsheet � Col's Tech

ZFS Cheatsheet � Col's Tech

Bug #571444 “Boot hangs and unable to continue when automount di... : Bugs : “mountall” package : Ubuntu

Bug #571444 “Boot hangs and unable to continue when automount di... : Bugs : “mountall” package : Ubuntu: Same here (using Ubuntu 10.04 server). One of my hard drives is formatted with JFS but I forgot to install jfsutils. Now when I try to boot it says:

fsck: fsck.jfs: not found
fsck: Error 2 while executing fsck.jfs for /dev/sdb1
linuxroot: clean, ...
mount: wrong fs type, bad option, bad superblock on /dev/sdb1
�������missing codepage or helper program, ...

mountall: mount /mnt/backup [650] terminated with status 32
mountall: Filesystem could not be mounted: /mnt/backup

And there the boot process is stuck. No shell, no login prompt, just nothing. Happens with "normal" boot as well as "recovery mode".

After installing "jfsutils" by using a live cd the problem has disappeared.

Seems 11.10 is also having the same issue. JFS is supported by the kernel, but the fsck util isn't installed by default and as such my boots hang. Time to install JFSUTILS package.

Monday, February 27, 2012

Uncomplicating the Complicated | Hardware Secrets

Uncomplicating the Complicated | Hardware Secrets

From 32 to 2 ports: Ideal SATA/SAS Controllers for ZFS & Linux MD RAID - Zorinaq

From 32 to 2 ports: Ideal SATA/SAS Controllers for ZFS & Linux MD RAID - Zorinaq

The PC Weenies - Review: MAXPower eSATA 6G PCIe 2.0 Controller Card

The PC Weenies - Review: MAXPower eSATA 6G PCIe 2.0 Controller Card

The SSD Review

The SSD Review

Highpoint Rocketraid 640 - FreeBSD 8.0 support and driver? [Archive] - The FreeBSD Forums

Highpoint Rocketraid 640 - FreeBSD 8.0 support and driver? [Archive] - The FreeBSD Forums: "interesting:

Conclusion (

While the Rocket 620 is a great option for those who don’t have a SATA 6Gbps port, or who want to maximise
their SATA 6Gbps performance, it does have limitations that are solved by the RocketRAID 640. Adding a second
drive when using the 620 resulted in heavy bottlenecking when both drives were used simultaneously, whereas we
saw great two-disk JBOD performance when using the 640.

However, the RocketRAID 640 brings its own problems, and while a two-disk JBOD setup works great thanks to the
dual Marvell 9128 controllers, adding a third or fourth drive in JBOD configuration will result in very poor

'via Blog this'

StarTech has disappointed me (PEXESAT32)

So today I loaded what I thought would be a lovely dual-eSATA port HBA expansion card (StarTech PEXESAT32) into my system.  The results were anything but lovely.  

First up, by default, I could only see 3 of the 7 drives I have in my eSATA enclosure, and I had no idea why.  I couldn't get the Ctrl+M key sequence in BIOS post to work, and ultimately figured out that my Asus Motherboard BIOS was preventing that (updated to a newer BIOS and now I can enter the Marvell setup area). 

Once into the setup area, I could only see 3 HDDs (as was showing up in Linux and in the post messages).  Naturally my first impulse was to check for firmware and bios updates for the card (at the time showing BIOS v1.0.0.1029, Firmware  Sadly StarTech had none.  Did some quick googling and found a nice site ( the Marvell 912x cards and linking to updated bioses.  Now mind you, this chip is used in a lot of aftermarket cards like this one, so it was relatively easy to find another manufacturer (a better one because of it) that posted a DOS executable to install the newer BIOS & Firmware.  Next problem?  Why the hell in the year 2012 are we still relying on DOS.  Seriously gentlemen, get with the year 2000.  Linux should be defacto now. 

Anyway, I managed to pull together a bootable USB from the following instructions:

Then copied in the update bits from the IOSATA site:

Rebooted, and booted from my USB.  Ran the go.bat, and after a little bit, all was updated and rebooted.  Now running BIOS and Firmware  Good right?  Wrong.  Do difference to the problem. 

I see someone bought this same card and having the same troubles also, so I guess I have a crappy product on my hands again.

Also, I'm a bit suspicious about Amazon's removal of the original page for the product that I bought.  It did have customer reviews there and now only a Google cache view will work.  Original Amazon product page:

Don't buy StarTech PEXESAT32.  Don't support this sort of poor product development.

And in case anyone is wondering, I just looked around for another card maker.  Syba seems to be another one of those StarTech vendor:  Cheap in most aspects.  Their website is worse than most knock-off computer product sales websites, and their support side doesn't even list a wide selection of products they sell.  I guess they're off my list of potentials now as well.

Wednesday, February 15, 2012

Batland � Mac OS X and dd

Batland � Mac OS X and dd

Sans Digital TowerRAID TR8M-B (wip)

Testing out this enclosure in an attempt to move to a ZFS 6-disk dual parity cluster.

WARNING:  THIS WILL BE A MESS OF A POST while I'm working through the various tests.  I will cleanup and organize my thoughts after I'm happy with the correct set of tests and order for this.

eSATA supports up to 5 devices via port multiplication, and the TR8M has 8 disk slots, split 4 disk slots per eSATA port (2), with a rather cheap insertion mechanism (but it seems to be working fine so far in my tests).  I have it temporarily connect directly to the sole esata port my motherboard offers

I also have acquired a StarTech PEXESAT32 card, based on a Marvel 9128 chip, for dual eSATA ports. When booting the system, the post message says to hit Ctrl+M to enter setup but I just can't get this to work ever.  Identified by the following in lspci:

0b:00.0 SATA controller: Device 1b4b:9123 (rev 11)

Motherboard is a Quad Core Intel i7 based, ASUS P7P55D-E Deluxe LGA1156 Intel P55 DDR3 - 2133 ATX.

First test was with a single 3TB Hitachi 5400-rpm disk loaded into position one.  I transferred, without interruption, an entire 3TB volume to it.

Second test, dd if=/dev/zero of=/dev/sdi bs=1M (where sdi is the device identifier for a second 3TB Hitachi 5400 RPM drive I have inserted into position 2 in the enclosure.  Currently humming along at a 71.4 MB/s rate, and we'll see what the rate looks like at the edge of the disk.

After 730GB transferred, rate is now 74.7MB/s.
After 2.5TB, rate is 88.8MB/s.
And finally, 3000592982016 bytes (3.0 TB) copied, 35449.4 s, 84.6 MB/s

Next, I formatted drive2 with ext4 and transferred 500+ GB.  Average at 500GB mark was 65.25MB/s.  For the next 100GB or so store rate was 65.8 MB/s averaged.

Using "rsync -av --delete-excluded " the results:
   sent 783607353796 bytes  received 2161398 bytes  63,029,118.46 bytes/sec

Ran another 260GB test copy:
   sent 264064015788 bytes  received 14133 bytes  69,591,258.38 bytes/sec

Looking solid.  Before I could do my next tests, I found drive slot 3 failed to work.  Since I picked this enclosure up at Frys and it was a markdown due to a return, I now figured out why it was returned.  Took it back and replaced with a new unit.  Also, I ordered some Syba cables (SY-CAB-ESA e-SATA to e-SATA Cable, 21-Inch) since some people have reported the cables that came with the enclosure are substandard.  So far my tests haven't shown any troubles with them, but I'll give the Syba cables a run for comparison.

Next test, using the Syba cables, dd from drive 2 (Hitachi 3TB) to drive 1 (WD 3TB), and drive 4 (Seagate 1.5TB) to drive 3 (WD 3TB) simultaneously.  Here's some early results:

   3745513472 bytes (3.7 GB) copied, 214.557 s, 17.5 MB/s
   3451912192 bytes (3.5 GB) copied, 206.387 s, 16.7 MB/s

   58474889216 bytes (58 GB) copied, 3477.81 s, 16.8 MB/s
   58770587648 bytes (59 GB) copied, 3486.02 s, 16.9 MB/s

   125004939264 bytes (125 GB) copied, 7447.71 s, 16.8 MB/s
   124681977856 bytes (125 GB) copied, 7439.6 s, 16.8 MB/s

   451648946176 bytes (452 GB) copied, 27096.5 s, 16.7 MB/s
   451353247744 bytes (451 GB) copied, 27088.4 s, 16.7 MB/s

   605514891264 bytes (606 GB) copied, 36413.1 s, 16.6 MB/s
   605839949824 bytes (606 GB) copied, 36421.3 s, 16.6 MB/s

Interesting my speeds are down compared to the early single drive test, but this isn't like for like yet.  Will have to retest with A) a single WD drive and B) a single Hitachi drive.

Let's try single drive reads.  Read from WD 3TB to /dev/null:

    29657923584 bytes (30 GB) copied, 242.405 s, 122 MB/s

This seems to continue around the same pace.  But when I start a read from TWO drives at once, speeds nearly cut in half combined: 

    drive1: 3714056192 bytes (3.7 GB) copied, 112.843 s, 32.9 MB/s
    drive2: 3716153344 bytes (3.7 GB) copied, 112.893 s, 32.9 MB/s

Same results using the PEXESAT32 controller as JBOD:

  Single drive only: 5524946944 bytes (5.5 GB) copied, 42.835 s, 129 MB/s
  Dual drive read:
     drive1: 3511681024 bytes (3.5 GB) copied, 55.2967 s, 63.5 MB/s
     drive2: 3511681024 bytes (3.5 GB) copied, 55.2987 s, 63.5 MB/s

That's a big improvement for the PEXESAT32 (Marvel 9128 based) versus the Onboard Marvel.

How about 3 drives on the expansion card?

   3764387840 bytes (3.8 GB) copied, 88.9512 s, 42.3 MB/s
   3764387840 bytes (3.8 GB) copied, 88.9532 s, 42.3 MB/s
   3764387840 bytes (3.8 GB) copied, 88.9552 s, 42.3 MB/s

Again, overall the eSATA port is managing to pull about 127-129 MB/s on the Expansion card no matter how many drives I spread it across.

What's 4 drives on the onboard Marvell 9123 reading simultaneously look like you ask?  Yep, more speed degradation:

   1883242496 bytes (1.9 GB) copied, 135.96 s, 13.9 MB/s
   1952448512 bytes (2.0 GB) copied, 136.024 s, 14.4 MB/s
   1929379840 bytes (1.9 GB) copied, 136.047 s, 14.2 MB/s
   1933574144 bytes (1.9 GB) copied, 136.051 s, 14.2 MB/s

That's a combined speed of 56.7 MB/s, a drop-off of ~53.5% from single drive reads!  NOTE TO SELF: Test my onboard mirrored drives within my server for speed comparison.

Kernel is 2.6.34-gentoo-r12, Quad-core system.  Not sure what sort of tweaking might need to be done to improve things.

Now, i needed to revisit the single drive write, this time with the WD 3TB drive for comparison to the Hitachi test earlier.
  # dd if=/dev/zero of=/dev/sdg bs=1M
  16183070720 bytes (16 GB) copied, 137.352 s, 118 MB/s
  2980546994176 bytes (3.0 TB) copied, 30124.9 s, 98.9 MB/s
  3000592982016 bytes (3.0 TB) copied, 30456.3 s, 98.5 MB/s

Quite a bit faster than the Hitachi, but let's keep in mind I'm now in a new enclosure (same brand and model though) and using Syba cables versus the no-name in that comes with the enclosure.  I will retest the Hitachi a final time to compare results, after letting this WD run through it's paces.

Next tests will be with the Western Digital 3TB Caviar Green drives, in position 5 and 6, where I will be using a PEXESAT32 PCIe dual-port eSATA controller.

I will continue to update this post as more data is collected.

Building the next generation file system for Windows: ReFS - Building Windows 8 - Site Home - MSDN Blogs

Building the next generation file system for Windows: ReFS - Building Windows 8 - Site Home - MSDN Blogs

Bitlocker Drive Space Anomaly

Bitlocker Drive Space Anomaly