3

Had a power failure and now my mdadm array is having problems.

sudo mdadm -D /dev/md0 [hodge@hodge-fs ~]$ sudo mdadm -D /dev/md0 /dev/md0: Version : 0.90 Creation Time : Sun Apr 25 01:39:25 2010 Raid Level : raid5 Array Size : 8790815232 (8383.57 GiB 9001.79 GB) Used Dev Size : 1465135872 (1397.26 GiB 1500.30 GB) Raid Devices : 7 Total Devices : 7 Preferred Minor : 0 Persistence : Superblock is persistent

 Update Time : Sat Aug 7 19:10:28 2010 State : clean, degraded, recovering Active Devices : 6 Working Devices : 7 Failed Devices : 0 Spare Devices : 1 Layout : left-symmetric Chunk Size : 128K Rebuild Status : 10% complete UUID : 44a8f730:b9bea6ea:3a28392c:12b22235 (local to host hodge-fs) Events : 0.1307608 Number Major Minor RaidDevice State 0 8 81 0 active sync /dev/sdf1 1 8 97 1 active sync /dev/sdg1 2 8 113 2 active sync /dev/sdh1 3 8 65 3 active sync /dev/sde1 4 8 49 4 active sync /dev/sdd1 7 8 33 5 spare rebuilding /dev/sdc1 6 8 16 6 active sync /dev/sdb

sudo mount -a

[hodge@hodge-fs ~]$ sudo mount -a mount: wrong fs type, bad option, bad superblock on /dev/md0, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so

sudo fsck.ext4 /dev/md0

[hodge@hodge-fs ~]$ sudo fsck.ext4 /dev/md0 e2fsck 1.41.12 (17-May-2010) fsck.ext4: Group descriptors look bad... trying backup blocks... /dev/md0: recovering journal fsck.ext4: unable to set superblock flags on /dev/md0

sudo dumpe2fs /dev/md0 | grep -i superblock

[hodge@hodge-fs ~]$ sudo dumpe2fs /dev/md0 | grep -i superblock dumpe2fs 1.41.12 (17-May-2010) Primary superblock at 0, Group descriptors at 1-524 Backup superblock at 32768, Group descriptors at 32769-33292 Backup superblock at 98304, Group descriptors at 98305-98828 Backup superblock at 163840, Group descriptors at 163841-164364 Backup superblock at 229376, Group descriptors at 229377-229900 Backup superblock at 294912, Group descriptors at 294913-295436 Backup superblock at 819200, Group descriptors at 819201-819724 Backup superblock at 884736, Group descriptors at 884737-885260 Backup superblock at 1605632, Group descriptors at 1605633-1606156 Backup superblock at 2654208, Group descriptors at 2654209-2654732 Backup superblock at 4096000, Group descriptors at 4096001-4096524 Backup superblock at 7962624, Group descriptors at 7962625-7963148 Backup superblock at 11239424, Group descriptors at 11239425-11239948 Backup superblock at 20480000, Group descriptors at 20480001-20480524 Backup superblock at 23887872, Group descriptors at 23887873-23888396 Backup superblock at 71663616, Group descriptors at 71663617-71664140 Backup superblock at 78675968, Group descriptors at 78675969-78676492 Backup superblock at 102400000, Group descriptors at 102400001-102400524 Backup superblock at 214990848, Group descriptors at 214990849-214991372 Backup superblock at 512000000, Group descriptors at 512000001-512000524 Backup superblock at 550731776, Group descriptors at 550731777-550732300 Backup superblock at 644972544, Group descriptors at 644972545-644973068 Backup superblock at 1934917632, Group descriptors at 1934917633-1934918156

sudo e2fsck -b 32768 /dev/md0

[hodge@hodge-fs ~]$ sudo e2fsck -b 32768 /dev/md0 e2fsck 1.41.12 (17-May-2010) /dev/md0: recovering journal e2fsck: unable to set superblock flags on /dev/md0

sudo dmesg | tail

[hodge@hodge-fs ~]$ sudo dmesg | tail EXT4-fs (md0): ext4_check_descriptors: Checksum for group 0 failed (59837!=29115) EXT4-fs (md0): group descriptors corrupted! EXT4-fs (md0): ext4_check_descriptors: Checksum for group 0 failed (59837!=29115) EXT4-fs (md0): group descriptors corrupted!

Please Help!!!

Improve this question
2
  • show /etc/mdadm/mdadm.conf Commented Sep 19, 2010 at 12:10
  • Had the exact same problem on a Raid 1 power outage. Commented Nov 24, 2012 at 20:02

2 Answers 2

Reset to default
1

From your description and the errors, it looks to me as though there are some serious data corruption issues. Remember, RAID protects against a very specific issue; limited disk failure. A power outage isn't protected against; that's why you use UPSes and keep backups as well as using RAID.

The one thing that looks odd to me is the inclusion of /dev/sdb instead of /dev/sdb1 in the list of RAID devices. Is that correct, or did the last character get cut off perhaps?

I would try the remaining backup superblocks, just in case.

Other than that, you might look for disk recovery software. Ideally you'll be able to take a backup image of the current state of the disks; that will reduce the chance that further changes will damage the data irreparably.

Improve this answer
2
  • Yes I agree :( - I tired all the superblocks but none worked - I tried using TestDisk - it can see the partition but most of the data on it is bad. I can see my folders but can't open them. - Cry:( Commented Aug 8, 2010 at 0:39
  • Yeah, sdb instead of sdb1 looks odd, you're right. @MatthewHodgkins, how is that? Commented Apr 15, 2012 at 4:42
0

You RAID setup had several flaws:

  1. RAID-5 with number of disks ≥ 3—4 is rather fragile. One disk's being kicked out and your data are in trouble.
  2. Not using write-intent bitmaps is dangerous and makes item #1 only worse.
  3. Spare could be more reasonable used as primary for RAID-6 or RAID-10 instead…

(I could add small chunk size and NOT using LVM-2 as disadvantages too, but they do not strongly affects overall status, of course.)

Now — never do anything to the array (fsck, and so on) until it's fully repaired. And I would strongly recommend you not trying to recover the data by yourself. You'd better find a specialist instead (in case you value them, of course).

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.