Post Reply 
 
Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
This one is a little hard to describe..
06-10-2016, 07:16 PM (This post was last modified: 06-10-2016 07:47 PM by redzot2009.)
Post: #1
This one is a little hard to describe..
So this has been going on for a few weeks now and I have replaced 3 drives and I just don't know what to do so I am turning to you folks to help me out.
  • Initially started with a bad drive. It was my last remaining 2TB disk so I figured what the heck I can just upgrade that one to a 4TB disk. We have Amazon Now so I did the needfull and it arrived in an hour. Yay! (or so I thought) (BTW this whole time the DroboPro is connected via iSCSI)
  • Once I put the 4TB disk in I knew the rebuild time was going to be atrocious as the total array size was going to be over 25TB at this point with 14TB or so in use. The drive initialized and started to rebuild with the dashboard reporting (I think) a 120 hour rebuild time. Honestly I didn't think much of it considering the size of disks and moved on about my day.
  • About 120 hours or so later I checked and the rebuild was finished however I was showing that another 4TB drive in slot 6 was bad. This didn't make me happy however no biggie, I have space and I will just order 4TB drive #2 and RMA the old one.
  • Another quick adventure with PrimeNow (this service is so freaking awesome!) and 4TB is in my hand. I replace that one and it says about 120 hours. Ok no biggie.. rinse and repeat.
  • This time I come back and it says I have a 3rd bad 4TB disk in slot 0. *****??? That drive isnt very old but... ok time for more drastic measures. This is where trouble really starts. The rebuild seems to finish but the drobo shuts off. when I power it back on it goes into some sort of rebuild mode and shuts off again. I call a buddy of mine up who has a DroboPro that he isn't using anymore and ask what he wants in trade. Long story short I have a gently used DroboPro in my rack. I followed the unit replacement procedures carefully from the Drobo FAQ. Powered it up connected via USB and bingo it starts to power up no problem.
  • Here is where it gets kind of funky. NewDrobo powers up and seems to be good connected via USB. After a few minutes the Dashboard goes into a rebuild mode. I think this is perfectly normal as a rebuild should be needed after a major drive transfer. *shrug* then it restarts for no reason. Seems fine. THEN all the lights start blinking. Drive lights are yellow. I let it sit and then it reboots after about 10 minutes.




Does anyone have any thoughts? Honestly if my wife's photography business were not on here I would probably just take a hit on the data as most of its old archive and not worry about it. But... this is pretty serious for her.

I would be happy to run the diags but I have no way to translate them. If one of you do I would be happy to send.

Thoughts? Please?

Sincerely,

Red

Edit: Gah! it did it agian. This time I made a video: DroboPro *****??
Find all posts by this user
Quote this message in a reply
06-10-2016, 07:40 PM
Post: #2
RE: This one is a little hard to describe..
hi red, here are a few thoughts...

over time, (and usage) drives can start to wear out in some way.
also, any intense process (such as during a rebuild or similar) could also make any drive on its last legs work hard, and in your case, possibly one of the drives was also a bit flakey during a 2nd rebuild (just as when you had a drive go bad for the first rebuild).

with multiple rebuilds, any other drive(s) that were starting to wear out in some way, probably were not able to cope with the extra work needed for the latest rebuild (and i think that is what is causing the lights and reboots)

when you mentioned that you started out with a bad drive (i wasnt sure what you meant, but after re-reading your 1st post, it seems like you already had a drobo pro with drives and data in/on it, and then simply added a drive that was a bit flakey, with a notion to upgrade/replace it later)


what you could do next, is to try putting your drobopro into ReadOnly mode, as mentioned here, as it has in some cases helped to stabalise a rebooting drobo, to allow acces to the data, in read only at least:
http://www.drobospace.com/forums/showthr...http://www.drobospace.com/forums/showthread.php?

(theres some other things to try though if you can try that when you get a chance, what happens now?)

(btw i have XP home SP2, a Drobo v1 with 2x 1TB/2x 1.5TB WD greens, & a bkp Drobo v2 with the same + a DroboShare: unused)
& a DroboS v2 with 3xWD15EADS &2x1TB in DDR mode on win7, & a drobo5D (all usb)
  • btw i did a sustained (write) operation for about 6 hours, and got 13.2MB / sec ...objection? "sustained" :)
    (16.7MB/s on a v2 & 47-96MB/s drobo-s)
Find all posts by this user
Quote this message in a reply
06-10-2016, 08:03 PM (This post was last modified: 06-10-2016 08:05 PM by redzot2009.)
Post: #3
RE: This one is a little hard to describe..
I tried the read only mode and it still rebooted. Waiting to see what it does.

The good news is that I can copy data off while it is up and running. 2TB of data... 15min at a time is going to take FOREVER...

Yes to answer your question I now have 2 DP units. Either several very flakey drives or some parity bit corruption some how.

I did take 2 of the drives I had that I thought were bad and used them to reset the old DroboPro I had. They come up like a champ now. so I think the issue is to many drives replaced over time.

The larger plan is that if I can get this unit stable I will remove all data and reset and reformat these drives. Then it might settle down and be trustworthy again.

Thanks for the reply. Check out my video as well.. might as well watch it I spent a few min doing it.
Find all posts by this user
Quote this message in a reply
06-10-2016, 08:30 PM
Post: #4
RE: This one is a little hard to describe..
thanks for the info, ok thats good at least theres a window for access
ah i didnt see the video link :)

(btw i have XP home SP2, a Drobo v1 with 2x 1TB/2x 1.5TB WD greens, & a bkp Drobo v2 with the same + a DroboShare: unused)
& a DroboS v2 with 3xWD15EADS &2x1TB in DDR mode on win7, & a drobo5D (all usb)
  • btw i did a sustained (write) operation for about 6 hours, and got 13.2MB / sec ...objection? "sustained" :)
    (16.7MB/s on a v2 & 47-96MB/s drobo-s)
Find all posts by this user
Quote this message in a reply
06-10-2016, 11:08 PM
Post: #5
RE: This one is a little hard to describe..
looks like RO mode had no positive effect
Find all posts by this user
Quote this message in a reply
06-13-2016, 01:18 PM (This post was last modified: 06-13-2016 01:28 PM by Paul.)
Post: #6
RE: This one is a little hard to describe..
thanks for the info and for trying that red,

aside from using the current windows of uptime (to at least access and copy anything urgent that your wife needs), then here are some more things you could try...

(some may say that while you have access to the data, at least in 15minute slots, it might be better to copy as much of the data for her as is needed before trying the things below)

its ok if no longer in readonly mode, but assuming that the drobopro is still in read only mode currently, then...

test 1) this is essentially to test the drobopro hardware (which is currently holding your disk pack) to check if it is still ok.
(if it does not have a problem when empty or after test 2, then the problem seems to be with 1 or more drives)

- to power all off (drobos and computer etc)
- to fully unplug all connections of, and to set aside, the other drobopro
- to take the currently problematic drobo pro (which has your diskpack) and to unplug all connection cables and power cables from it
- now without any power or cables attached, to then remove all drives (remembering the order of your disk pack for later)
- to then power up the problematic but empty & disconnected drobopro
(does it boot up and then go into standby mode?)

- to then power up the computer, and when that is running, to launch dashboard
- to then connect the drobopro directly to the computer
(does it wake up shortly after, and then become recognised by dashboard, probably with a red light and message about putting a drive in? - if so please DONT).

- to then power all off again
- to unplug all power and connection cables
- to then (with power still OFF) to put your disk pack drives back where they were.
- to then power up the unconnected drobopro
(does it boot up with blue led sequence, ultimately lighting them all up from 1 through to 10, and then going into standby mode?)

- even if not, to then power up the computer and dashboard
- and to then connect the drobopro back to the computer
(does it wake up shortly afterwards and then become recognised by dashboard, and computer?)

what happens now red? does it still show read only, but still reboot and loose connection with the computer access and dashboard?

(by the way, are all other unneccessary programs closed, and are there any possibly automatic scheduled tasks also trying to antivirus scan the drobo or index the contents?)


2) the 2nd test would be similar to the 1st test, but would essentially be to test whether the creation of a fresh new disk pack (using other spare blank drives if available) works fine, or if it still reboots and has problems... (if no problem are there then i think its most likely drives related and a 3rd option might be needed)

This would involve shutting all down, and totally removing the other drobo from the equation (as above),
and then to remove all disk pack drives from the problematic drobopro (remembering the order of the drives, and known as diskpack1 ), and then setting up a fresh set but i will post steps below for you even though some are similar to above.

- to power all off (drobos and computer etc)
- to fully unplug all connections of, and to set aside, the other drobopro..
- to take the currently problematic drobo pro (which has your diskpack) and to unplug all connection cables and power cables from it
- now without any power or cables attached, to then remove all drives (remembering the order of your disk pack for later, and likely memorising them as "diskpack1")
- to then power up the problematic but empty & disconnected drobopro
(does it boot up and then go into standby mode?)

- to then power up the computer, and when that is running, to launch dashboard
- to then connect the drobopro directly to the computer
(does it wake up shortly after, and then become recognised by dashboard, probably with a red light and message about putting a drive in?)

at this point, if the drobo is still found in this way, that is good, and then please can you come out of readonly mode, and back into Normal mode.
- then if the empty drobo pro is still found in this way (and no longer in readonly mode), then you can try inserting a good working blank drive (or a drive that you do not mind being wiped), but the drive must not be part of a disk pack. (if the drobo accepts and assimilates this drive as a new pack, then ideally if you have a 2nd good spare/blank/un-needed drive, you could add that as well.
- these 2 drives would esentially now be your "diskpack2".
- to then try copying several GB of data onto the newly created diskpack2 volume, and to try and use it as you normally would, (such as accessing the copied data, changing it, moving it etc etc) to see if it remains up as though working fine.

if so, then it looks like the drobo pro is working fine with the drives that make "diskpack2".
(ideally, having a full compliment of drives, or testing the usage of all slots in certain ways would help eliminate slot or backplane issues, but seeing as you already tried migrating to this 2nd unit already, it seems more likely to be a problem with the diskpack1 drives.)


After testing diskpack2, to get back to diskpack1,
- please power all off and disconnect all cables and power from the drobo
- and then with power still OFF, to remove all drives which are "diskpack2", and then to put all drives from "diskpack1" back inside.
- and then to power up the drobopro and computer how you normally would.


before the 3rd) option, if you are able to try to capture the drobo and then dashboard in another set of youtube clips, they could be handy, (and also when in dashboard, if you could try clicking on each drive slot to see if any other messages or dropdowns appear, it may have some more info too).

another option of course would be to continue backing up at least the most important data for you wife, and then raising a ticket for the support team (even if at a cost if not in warranty), as they may be able to advise you on which problematic drives exist that might need special cloning, based on some logs they may ask for, but please dont try taking logs just yet because the generation process might cause some issues during copying)

also, after test 1) is carried out, it will also see whether reseating all drives helps, just in case a drive was making an intermittent connection in some way.

(sorry for long post)

(btw i have XP home SP2, a Drobo v1 with 2x 1TB/2x 1.5TB WD greens, & a bkp Drobo v2 with the same + a DroboShare: unused)
& a DroboS v2 with 3xWD15EADS &2x1TB in DDR mode on win7, & a drobo5D (all usb)
  • btw i did a sustained (write) operation for about 6 hours, and got 13.2MB / sec ...objection? "sustained" :)
    (16.7MB/s on a v2 & 47-96MB/s drobo-s)
Find all posts by this user
Quote this message in a reply
06-19-2016, 09:00 PM
Post: #7
RE: This one is a little hard to describe..
Hi! Sorry for the delay but work/family dont give lots of time for stuff like this. I tried everything you reccomended but it would follow the same pattern. I even got one of the 4TB RMA'd disks back and no change. Good news though... I did try one thing that keept the Drobo on (at least for now). I changed the protection level to Dual Disk redundancy and it wants me to replace what it thinks is the bad drive. I tried that and it shut down... so I rebooted with the old "Bad" 4tb disk in it and it stayed on and I can access the data. Currently, slowly, pulling pictures/video/documents off the array. Fortunately I have acquired a Dell DX60125 that I have been able to convert back to a Dell R510 and it has 12x 3tb drives and 2x 300gb 15k drives in it so I have a ton of space to distribute data to.

Next up is to remove all data from the Drobo and totally reset it/add the 4TB drive in to replace the smallest drive (3TB) and then figure out what to do next. I have a crap ton of storage in these rack servers and plenty of replacement parts (work) if i need them for the future. Once I get this step done I will update yall on what happens.

Thank you for the tips and things to try, I never would have thought about a RO mode on one of these so great suggestions.
Find all posts by this user
Quote this message in a reply
06-20-2016, 02:47 PM (This post was last modified: 06-20-2016 02:47 PM by Paul.)
Post: #8
RE: This one is a little hard to describe..
thanks red am glad you managed to get it more stable.
once you do the reset, it is probably worth using ddr again, especially as any future drive errors or rebuilds will take longer as the amount of stored data increases over time.

one thing just to bear in mind though, is that when you reset, it might be worth surface scanning the drives indiviually before using them again (or if the ones you will use are ok, then one thing i try to do is to hook up any new drives to a standalone usb caddy and surface scan it, before i start using them in my drobo or computer)

(btw i have XP home SP2, a Drobo v1 with 2x 1TB/2x 1.5TB WD greens, & a bkp Drobo v2 with the same + a DroboShare: unused)
& a DroboS v2 with 3xWD15EADS &2x1TB in DDR mode on win7, & a drobo5D (all usb)
  • btw i did a sustained (write) operation for about 6 hours, and got 13.2MB / sec ...objection? "sustained" :)
    (16.7MB/s on a v2 & 47-96MB/s drobo-s)
Find all posts by this user
Quote this message in a reply
06-25-2016, 04:32 PM
Post: #9
RE: This one is a little hard to describe..
yeah i was planning on doing that.

honestly it will be tough to go back to the drobo. the replacement server for now is an R720XD full of 3TB drives
Find all posts by this user
Quote this message in a reply
06-27-2016, 05:46 PM
Post: #10
RE: This one is a little hard to describe..
ok sure thats always up to you... i guess if things had been working ok for you for a long time and then a glitch happened now, it could just be down to old age somewhere in the mix. (or components with lots of life experience to be politically correct) :) and that maybe a full reset could see many more years to come or possibly a newer unit with the extra warranty etc that it could bring. either way, am glad you didnt lose access to any data, which is the main thing in the end.

(btw i have XP home SP2, a Drobo v1 with 2x 1TB/2x 1.5TB WD greens, & a bkp Drobo v2 with the same + a DroboShare: unused)
& a DroboS v2 with 3xWD15EADS &2x1TB in DDR mode on win7, & a drobo5D (all usb)
  • btw i did a sustained (write) operation for about 6 hours, and got 13.2MB / sec ...objection? "sustained" :)
    (16.7MB/s on a v2 & 47-96MB/s drobo-s)
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump: