0

I'm building a simple ZFS file server for the small business I work for. The server is a Dell Poweredge 840, with 1GB RAM. The OS (OpenSolaris 2009.06) is installed on one SATA drive, and there are three other SATA drives installed for storage: 1x1TB, 1x1.5TB, and 1x2TB. When I add the three drives to one raidz zpool, throughput isn't very good:

#zpool create -m /export/pool pool raidz c7d1 c8d0 c8d1 #zfs create pool/fs #time dd if=/dev/zero of=/export/pool/fs/zerofile bs=1048576 count=1024 1024+0 records in 1024+0 records out real 0m12.539s user 0m0.002s sys 0m0.435s

That's about 81.6 MB/s. That's not horrendous, but I tried creating a pool consisting of just one of those drives:

#zpool create -m /export/disk-c7d1 disk-c7d1 c7d1 #zfs create disk-c7d1/fs #time dd if=/dev/zero of=/export/disk-c7d1/fs/zerofile bs=1048576 count=1024 1024+0 records in 1024+0 records out real 0m21.251s user 0m0.002s sys 0m0.552s

Okay, 48.19 MB/s throughput for a sequential write to one drive? That seems pretty low. Especially when I format the drive as UFS and try that same write:

#newfs /dev/dsk/c7d1s2 <snip> #mount /dev/dsk/c7d1s2 /mnt/c7d1 # time dd if=/dev/zero of=/mnt/c7d1/zeroes bs=1048576 count=1024 1024+0 records in 1024+0 records out real 0m10.372s user 0m0.002s sys 0m1.720s

That's almost twice the speed, 98.73 MB/s. That's much closer to what I'd expect out of these drives (though they're just cheap SATA drives).

What am I doing wrong here? I understand that there's overhead involved in writing parity data with RAIDZ, but making a pool from a single drive shouldn't halve throughput, should it? That seems pretty bad.

Thanks, everybody.

Improve this question
1
  • Something ain't right... I've got 4 older SATA drives in a RAIDZ and they pull 180MBps easy. Newer drives with larger caches and better all around technology should be able to easily surpass that. Commented Apr 20, 2011 at 2:35

3 Answers 3

Reset to default
2

ZFS performance can be greatly improved by adding read and write cache SSDs (i know i know they're pricey) Check this link for more info.

Improve this answer
2
  • Thanks, but this is really a cheap, bare-bones setup, and I'm just trying to squeeze performance out of the box as it is now. The machine will be dealing exclusively with very large sequential reads and writes (200-1000GB), so a SSD cache probably wouldn't be very helpful. Commented Apr 19, 2011 at 16:11
  • 3
    If you are trying to squeeze performance out of it (instead of space), use mirrors. Commented Apr 19, 2011 at 20:37
2

If you want performance, stay away from raidz and use mirrors instead.

"Should I Configure a RAIDZ, RAIDZ-2, RAIDZ-3, or a Mirrored Storage Pool?"

"WHEN TO (AND NOT TO) USE RAID-Z"

Improve this answer
1
  • Right, I expected some performance hit from RAIDZ. I still don't quite understand why throughput to a single disk would be halved by ZFS, though: when I created that single-disk pool, obviously it wasn't RAIDZ. Commented Apr 19, 2011 at 21:03
0

Keep in mind, with ZFS there is more to every write, because some of the data is metadata. What's interesting is that if you are writing async with dd, your performance should be better, since there should be some caching going on. As you begin to scale-up pools, you will see improvements in performance, that will definitely be amazing if you do through SSDs as cache devices, and as ZIL devices, assuming you have a lot of sync writes. Cache SSDs are your read cache, and ZIL SSDs are your sync write cache. First thing I would try, though your setup is clearly not ideal, is to use a blocksize that matches your filesystem's blocksize, which is likely 128K. See what that will look like.

Lastly, do not compare ZFS and UFS - apples and oranges. Others provided excellent references on this exchange. Certainly worth a read.

Improve this answer
2
  • Well, that makes sense. Is the additional metadata hit sufficient to explain the halved throughput? I suppose if there were a seek operation for every write, that could make sense? Commented Apr 21, 2011 at 3:15
  • Whoops, didn't mean to post so early. I suspected that my UFS test case was not exactly a valid comparison, but I wanted to get an idea of the maximum throughput of the drive without ZFS involved. I'd read the ZFS Best Practices guide before, but I'll look through it again. Thanks for your help. Commented Apr 21, 2011 at 3:16

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.