3

this is on nexenta open storage. I have a zpool that is in UNAVAIL state because of some disks. 

# zpool import -F pool: delta id: XYAZA state: UNAVAIL status: One or more devices are missing from the system. action: The pool cannot be imported. Attach the missing devices and try again. config: delta UNAVAIL insufficient replicas raidz2-0 UNAVAIL insufficient replicas c0t5000C500573978EBd0 ONLINE c0t5000C50057397933d0 ONLINE c0t5000C50057397C47d0 UNAVAIL cannot open c0t5000C5005739997Fd0 UNAVAIL cannot open c0t5000C50057399F87d0 ONLINE c0t5000C5005739A90Bd0 ONLINE c0t5000C5005739ABC7d0 UNAVAIL cannot open c0t5000C5005739AC0Fd0 ONLINE raidz2-1 DEGRADED c0t5000C50057398657d0 ONLINE c0t5000C50057398E03d0 ONLINE c0t5000C50057398EF7d0 ONLINE c0t5000C50057398FAFd0 ONLINE c0t5000C5005739A39Fd0 UNAVAIL cannot open c0t5000C5005739AAFFd0 ONLINE spare-6 DEGRADED c0t5000C5005739AC3Fd0 UNAVAIL cannot open c0t5000C50057398EDBd0 ONLINE c0t5000C5005739AE9Bd0 ONLINE ~# zpool import -FXf delta cannot import 'delta': I/O error Destroy and re-create the pool from a backup source.
  1. How can I get this zpool import work fine?
  2. Are the disks that are pointed out 'UNAVAIL', how can I get them repair/replaced?

Update: 7 Sep, 2018

After finding the disks to slot mapping on Nexenta, reseating the disks in delta volume, I could get one disk ONLINE in raidz2-1 group but the 3 disks in raidz2-0 group still continue to show as UNAVAIL.

# zpool import -F pool: delta id: XYAZA state: UNAVAIL status: One or more devices are missing from the system. action: The pool cannot be imported. Attach the missing devices and try again. config: delta UNAVAIL insufficient replicas raidz2-0 UNAVAIL insufficient replicas c0t5000C500573978EBd0 ONLINE c0t5000C50057397933d0 ONLINE c0t5000C50057397C47d0 UNAVAIL cannot open c0t5000C5005739997Fd0 UNAVAIL cannot open c0t5000C50057399F87d0 ONLINE c0t5000C5005739A90Bd0 ONLINE c0t5000C5005739ABC7d0 UNAVAIL cannot open c0t5000C5005739AC0Fd0 ONLINE raidz2-1 DEGRADED c0t5000C50057398657d0 ONLINE c0t5000C50057398E03d0 ONLINE c0t5000C50057398EF7d0 ONLINE c0t5000C50057398FAFd0 ONLINE c0t5000C5005739A39Fd0 ONLINE c0t5000C5005739AAFFd0 ONLINE spare-6 DEGRADED c0t5000C5005739AC3Fd0 UNAVAIL cannot open c0t5000C50057398EDBd0 ONLINE c0t5000C5005739AE9Bd0 ONLINE

I found a single spare disk for another online pool on the appliance. Now the question is 1. Should I detach/attach the spare to raidz2-1 group which has only one disk UNAVAIL under sub category of spare-6 ? or 2. Should I detach/attach the spare to raidz2-0 group which still has 3 disks UNAVAIL ?

Given that this pool is offline (after export) and is not importable now.

I looked at some history on how this pool was created, it was created with the below command in the past

zpool create -m /volumes/delta -o autoreplace=on delta raidz2 c0t5000C500573978EBd0 c0t5000C50057397933d0 c0t5000C50057397C47d0 c0t5000C5005739997Fd0 c0t5000C50057399F87d0 c0t5000C5005739A90Bd0 c0t5000C5005739ABC7d0 c0t5000C5005739AC0Fd0 raidz2 c0t5000C50057398657d0 c0t5000C50057398E03d0 c0t5000C50057398EF7d0 c0t5000C50057398FAFd0 c0t5000C5005739A39Fd0 c0t5000C5005739AAFFd0 c0t5000C5005739AC3Fd0 c0t5000C5005739AE9Bd0

Does that indicate raidz2-0 and raidz2-1 are two RAIDZ mirror pools?

If yes, should I prioritize fixing raidz2-1 group of disks to become online over raidz2-0 (which has 3 disk failures)?

Improve this question
6
  • 3
    You have a 3-disk failure on raidz2. Where are the disks? If they are unplugged, then plug them back in. If they have failed, then your pool is dead, and you can only restore from backup. Commented Sep 6, 2018 at 22:19
  • how do I know if they are failed? fmadm command on nexenta shows the disks could have had some temperature related failures Commented Sep 6, 2018 at 22:37
  • @NikhilMulley Check SMART status logs, check/re-seat connections (cables, controllers). Are all disks of the same type/manufacturer/date? Commented Sep 7, 2018 at 7:36
  • hi @user121391 thanks for the comments. Unfortunately most of these disks do not have SMART enabled for them, however fmadm did show some of them had some high disk temperatures. I tried finding the disk to slot mapping on nexenta, reseated all of those disks and was able to get one disk under raidz2-1 group online but rest of them still show UNAVAIL. 3 disks in raidz2-0 are still continuing to show as UNAVAIL Commented Sep 8, 2018 at 0:20
  • I have updated with new set of information but still have some open questions. Any pointers/hints/suggestions/fixes are please welcome. Thanks again. Commented Sep 8, 2018 at 0:27

1 Answer 1

Reset to default
1
  1. Should I detach/attach the spare to raidz2-1 group which has only one disk UNAVAIL under sub category of spare-6 ?

Your spare disk has already been automatically attached to raidz2-1 and has been resilvered. Normally, you would now detach c0t5000C5005739AC3Fd0 and remove it form the system because it has failed once (or repurpose it on another system and hope for the best, if you don't have any resources to spare).

But that would not help you, as raidz2-0 is still down. If any vdev in a zpool fails, your whole pool and all data on it is lost. Each vdev will fail if the configured parity cannot be upheld, because enough disks have failed inside it. In your case, each Z2 vdev can sustain the loss of two disks, but one of them has lost three.

  1. Should I detach/attach the spare to raidz2-0 group which still has 3 disks UNAVAIL ?

First, you have no real spare now. And even if you removed a working disk from raidz2-1 (not recommended!), you would still gain nothing until you fix your failed disks in raidz2-0. The parity data has to come from somewhere, and in this case it has to come from your failed disks. If you don't get a t least one of them to work, your best bet would be to restore from backup.

I looked at some history on how this pool was created, it was created with the below command in the past zpool create -m /volumes/delta [...] Does that indicate raidz2-0 and raidz2-1 are two RAIDZ mirror pools?

No, that means your pool consists of two striped vdevs, each configured with raidz2 redundancy and 8 disks. You can lose 2 disks in each vdev at most. If you lose 3 in any of them, it's game over.

In ZFS, all redundancy is handled at the vdev level. The pool is always striped over all existing vdevs. If any vdev fails, the pool fails. Mirroring is a type of redundancy for a vdev and is incompatible with raidz1/z2/z3 (you can mix and match vdevs with different redundancy types, but the weakest one will define the strength of your pool and the performance will suffer, so this is not recommended).

If yes, should I prioritize fixing raidz2-1 group of disks to become online over raidz2-0 (which has 3 disk failures)?

No, you must first fix raidz2-0.

All of them belong to Seagate, there is a mix of SEAGATE, Rev. A001 and SEAGATE, Rev. 0002, size 1.82TB

This doesn't help you now, but maybe in the future: It is advisable to mix and match manufacturers and models and even manufacture dates, to even out cases of "a bad batch of drives". For example, if you bought 4 disks of each of the 4 major manufacturers (Seagate, Western Digital, Hitachi, Toshiba) and split them 2-2 on your vdevs, even if all disks of one manufacturer failed at the exact same time, you would still have a working pool.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.