Friday, 24 April 2015

Flash Dell Firmware into Seagate Generic Drives (ST3450856SS to HS11 Firmware update, maybe others)




Hello,

I am building a server for one of my small business clients, it's a Dell PowerEdge 2950 with a Perc 5i RAID controller on board. This is an older server, but it is plenty powerful for what I need out of it. The idea is that this server will be used as an iSCSI target for an ESXi host. I don't know where the server initially came from, but when I got it from the client to build out, it only had 3 hard drives in it of various sizes, nothing big enough to be a storage server. I had a collection of old ST3450856SS hard drives from an old HP MSA2000 storage frame which were SAS drives of 450GB. I got the additional drive cages as required and mounted 6 of these old HP SAS disks into the server. The Perc 5i controller picked up on them no problem, and I was able to create a RAID5 set and off I went...however, I soon discovered some issues.

Now Perc 5i created the drive array no problem, it was accessible, but when I tried to run a CrystalDiskMark benchmarking tool on the server, the controller would drop out with Windows Event error 129, Reset to device, \Device\RaidPort1. The read portion of the software was fine, but it seems that as soon as it switched to Write tests, it would die, sometimes right away, at other times a few runs in. I started looking for a solution and updating everything, and one of the things that I wanted to update was the hard drive firmware. I looked up model number  ST3450856SS + firmware on google and found that Dell also uses these drives for some of their servers, so I thought I will just flash the newest firmware from Dell into these and it will be happy...but this was not so easy...actually it was not easy at all.

Now, on this Dell server I installed the Dell Openmanage Server Administrator and it showed that the disks were detected as ST3450856SS with firmware version 0007, and that the latest firmware available for these types of disks from Dell was HS11.











I downloaded the Windows firmware update packages called SAS-Drive_Firmware_288PJ_WN32_HS11_A07.exe, assuming that I would be able to just run the program and it would update all the drives, however, this was not the case. The package does some validation to see if the drives you have will actually work with this firmware, and returned a message:

This update package is not compatible with your system configuration

Great, so I stared looking online to see what this is, and there isn't much information other then to ensure that you have the latest drivers and such from Dell, I do, and none of it worked.

I started working to see how I could circumvent the firmware process, and the first thing I did was to extract the firmware update package exe file with UniExtract,  This gave me something like this:


Under the payload folder there was the actual firmware file, HS11.fwd. In the root was a file that seems like it would be responsible for the actual loading of the firmware called SASDUPIE.exe, so I did a /? on this executable under CMD and came up with the following path for the firmware update:

SASDUPIE.exe -u -s "%cd%\payload" -f -o update.xml -debug debug.log

When I ran the above command, it finished running very quick, must faster then would be required to upgrade the firmware on 6 SAS disks, so I looked at the debug.log file it create for some clues.

The first clue was the detection process and what it was looking for. The Seagate drives I have were being detected as Hardware ID of 00000, whereas the firmware was looking for 3 different Hardware IDs, corresponding to three different drive sizes this one hardware package could upgrade. So somehow, I had to trick the firmware into thinking that 00000 that it is detecting is actually the ST3450856SS hard drive. Here is the line in the debug.log that shows the detection process. I didn't get to capture the previous log, but here is what it looks like when it makes a successful detection, notice the 00000 being detected as the right model number:


I used the HEX editor to open the HS11.fwd firmware file and look inside. During my research into the topic of flashing the firmware, I read that a Dell version of the firmware would carry a Dell specific header in the first 256bits of the file, and this header is all that differentiated the firmware file from a generic Seagate firmware release. When I opened the file in HEX editor, this is what I saw first at the top:


As you can see, there are three sets of numbers on the side, 15305, 15304 and 15303. Each of those numbers is what we saw earlier in the debug log, and these numbers are followed by the model number of the hard drive. For me, 15305 and the model number ST3450856SS is of importance, so I busted out the "Replace" feature in my Hex editor and replaced the text 15305 with 00000. I saved the edited file over the original and tried to run the SASDUPIE.exe command again as I did previously to look at the debug log. This time, I got a message that there is a version check error, and that to install HS11, you need to have HS06 first, but my drives were reporting the firmware level of 0007 (from HP), so I had to do more modifications with Hex. As yo can see in the previous image, right at the very first line, there is a line of DELL.HS11HS06, so I replaced HS06 with 0007 and saved the file again. This is what the file looked like after the changes:



Upon running SASDUPIE.exe command again, it took considerably longer to complete, however, despite now matching the model number and firmware revision, it still failed. I was sensing that I was getting close, and thought that perhaps I need to flash the older version of the firmware first, but it was the end of the day for me so I left it for the next day. 

The following day I tracked down the HS0F firmware (F is higher than 6 in Hexadecimal, and HS06 is the minimum required, but I couldn't find that firmware easily) and opened it again in a Hex editor. 

This time, I made the same changes as before to HS0F firmware, replaced HS06 with 0007 and 15305 model number with 00000. At this point, I think it would work if you were to save the file into the payload directory and run SASDUPIE.exe command again, however, I didn't try this, as I already moved on to a different firmware upgrade process, but in hindsight I think it would have worked right from windows. 

Dell makes a downloadable utility called the "Dell SAS Hard Drive Firmware Utility" which can be installed in Windows, and then used to create a bootable USB drive to do firmware updates without windows. I ran the utility and it extracted the file contents to C:\Dell\Drivers\RG1GN\. If you navigate further to C:\Dell\Drivers\RG1GN\files\fw\sas\seagate\15K6 you will find all the firmware that is available for the particular Seagate drive. The files found here for HS0F and HS11 are the same files that you would have downloaded earlier, all in one place. I deleted all the other firmware versions and just left the HS0F.FWD file in there, the one I modified earlier to look for 0007 firmware. From this point, if you backtrack to C:\Dell\Drivers\RG1GN\ there is a utility called dddp.exe, run it and click on Install to a USB Flash drive, plug in a USB drive, and let it quickly build.


Once I booted off the USB into the interactive mode, it found my drives and said the firmware update was available. I clicked on Update on each drive (it locks your mouse, so don't freak out, takes about 5 min to read S.M.A.R.T. information twice, you can see the countdown at the top) and to my surprise, it went! I did this to all 6 disks! After this was all done, I went back into Windows, and put the previously edited HS11.fwh file in and recompiled the boot USB, however, it failed, but I knew why! 0007 version no longer exists on these drives, they are all now HS0F! Going back into the HEX editor, I changed the HS11.fwh file again and adjusted the header to it's original state of DELL.HS11HS06. I rebuilt the USB boot file again, and this time, success! All drives were updated to the latest from Dell, HS11! Woo hoo! Unfortunately, it didn't solve my Perc 5 errors in Windows, but at least I knew the drives were not the contributing factor, at least not on the firmware level. 


I realize that this is very specific to one version of the drive, but I believe you can use this approach probably with most firmwares, and perhaps not only from Seagate, but I haven't tested that theory yet. As long as the firmware appears to have a Dell header, using Dell tools to update the firmware should be just a matter of making similar changes. 






19 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. Really nice article. I am looking at something similar but from the opposite perspective. I was able to use the dell firmware utility to flash my "Dell" branded SAS drives in an MD1200 enclosure, however have two non Dell drives that it won't flash because the firmware doesn't match. So I don't want to flash the drives with Dell firmware, but use the Dell tool to flash the non-Dell drives with standard Seagate firmware. There really aren't any other good tools out there to flash these files within an existing array. I was thinking if I took the LOD file from Seagate with the firmware and plugged in the top block you reference above with HexEdit into the original Seagate firmware file and change the 15305 with 00000 may take. Any thoughts on if this would work?

    Thanks.

    Steve

    ReplyDelete
    Replies
    1. Hi Steve, sorry I don't know how do it it the other way around. I have only really done this once for this case. My issue at the end of the day was a bad Perc 5i controller and not the drives themselves, although having them at Dell firmware version made it look clean under server manager with checkmarks across the board. I am happy to report that these drives are working away without issue still, and I flashed about 5 more for spares.

      As for Unknown, your drives seem much newer then stuff I was dealing with, sounds like 2TB, so something probably isn't fully compatible with newer drives. My writeup was for 450gb drives which are pretty old.

      Delete
  3. This is a really great article, I commend you for the technical detail.

    I recently had a series of Dell OEM ST32000444SS SAS drives go belly up. Unfortunately, the generic Seagate ST32000444SS SAS drive would not work in my enclosure with my array because it was not a "certified" drive. The "certified" drive from Dell is 3x the cost of the Seagate off Amazon.

    I thought perhaps flashing the Dell firmware onto these drive would solve my problem. I looked long and far for someone who attempted to do the same as I. I came upon your article.

    I am sad to say, though, I was not successful in my quest using your instructions. I did download the update utility you mention, and the firmware for my drives was present in the directory tree.

    I attempted to flash on several platforms -- an R410 with a SAS6i controller, and a 1950 with a PERC6i and PERC5i SAS controller.

    The SASUPDIE utility, even after editing the model number and minimum version in my new firmware file, would apparently attempt to flash but then get an error returned from the drive (according to information in the debug log, "SASHardDriveDUPDevice::writebuffer 1: GOOD RC: but non-zero SCSIStatus=(unprintable character)"

    Using the firmware update utility was just as futile. The utility would report the drive as needing an upgrade, and show the "new" version available, but attempting to flash it would return a "error code 4400:281d invalid request".

    Heck, i went as far as pulling the circuit board off a failed Dell system which had already been upgraded and attempted to stick it on the Seagate drive -- this was an epic fail too, the Seagate drive with the Dell controller would time out on all SAS operations.

    I congratulate you on your success. In my case, I am guessing Seagate changed something on the "newer" ST32000444SS drives, as my Dell OEM versions are reported by SAS controllers as 1863 GB while the Seagate drive is only showing as 1823 GB. I had to break down and order Dell OEM drives, which, even "refurbished", cost more than Seagate drives new.

    ReplyDelete
  4. Fairly recent article but not sure if it's being monitored. Do we know if the above was successfully done what it would mean if a drive failed? Would Seagate do a warranty repair on it? Guessing if the drive failed couldn't flash it back to standard.

    For Unknown, surprised your drives don't work as is. Several years ago there were articles that Dell was accepting non Dell drives in their enclosures. Mine works in my MD1200 which didn't several years ago. Even my SATA drives work but always complaining about something in my monitoring software.

    ReplyDelete
  5. Hi, this article is awesome. I have a very similar issue (error 129 with hitachi SAS disks in a Dell T7500 perc 6/IR). How did you solve error 129 issue?

    ReplyDelete
    Replies
    1. Hi. Sorry for the delay, was on vacation. My error was actually not related to the drive firmware at all it seems, my raid controller was the actual issue. I ended up ordering a new PERC 6 controller off e-bay and all those issues went away. The server I was using had a PERC 5 in it originally but I was able to install a PERC 6 and the issue was no longer present. Sorry it doesn't seem to help you as you are already on a PERC 6 controller.

      Delete
  6. Any recommendations for loading firmware on to a drive with a "newer" revision?

    I have a retail Seagate ST1000NM011 drive with Seagate firmware SN03

    I need to load the Dell firmware PA09 on the drive. I followed alot of your steps. I eventually got the drive to attempt putting the firmware on using the Nautilus utility from Dell. The problem is the software sees that SN03 is a newer version than PA09.

    I event tried the step to modify the header of the PA09 firmware file from: DELL.PA09PA00x to DELL.PA09SN03x

    Any thoughts on what I could try?

    ReplyDelete
  7. very nice article, thank you for sharing this!

    ReplyDelete
  8. Very good info for the tinkerer. I appreciate it!
    I have a few Dell branded ST1000NM0011 drives as well and they are at PA09 revision. I need to get them to PD0A. Would your method apply to getting them to the needed level?

    ReplyDelete
  9. The better question is: can that drive accept the different firmware version? Meaning, will the drive that currently has PA09 accept, and not brick, the PD0A firmware?

    ReplyDelete
  10. Good article, cheers - only one I've been able to find close to a solution!

    Unfortunately I've not been able to succeed so far with ST1000NM0023 drives. Have been able to make the changes as suggested to the point where the debug log shows it accepting the drive and firmware version as correct - but unfortunately then just gives a failed to update error.

    I think the key here is as you found, you must use an old enough version of the firmware that it has no firmware pre-requisites. I have GS10, but I think I need GS00 (or close), and have been unable to find this version of the firmware update file thus far.

    ReplyDelete
  11. I am attempting to do this on some ST9600204SS drives. Following your directions to a T the debug log shows the following....If you could provide and ideal it would be very helpful.
    SASHardDriveDUPDevice: Strip Header Set
    SASHardDriveDUPDevice: Online Download Enabled
    m_D_Method = 1
    Image version = FM0A
    Min Image version = A709
    m_ProcessTime(byte) = x
    m_ProcessTime = x
    # of devices = 1
    deviceID [03
    deviceName [0 S0QH600
    SASHardDriveDUPDevice: Image version set: FM0A
    DUPWrapper: populate rc=f
    Comparing device ID:3 with 00000
    depCheck: NO Find matching device
    DUPWrapper:No devicea updated.
    DUPWrapper:Deleting ImageLoader.
    SASHardDriveDUPDevice::~SASHardDriveDUPDevice::Deleting m_PR_stopped
    SASHardDriveDUPDevice::~SASHardDriveDUPDevice::Finished deleting m_PR_stopped
    SASHardDriveDUPDevice::~SASHardDriveDUPDevice::Deleting m_PR_disabled
    ImageFiles::Finished deleting m_PR_disabledr

    Thank you
    Jerry Shelton

    ReplyDelete
    Replies
    1. Just looking at the log, it almost looks like it isn't detecting the drive? Sorry I did this a while ago, documented as much as I could and only what worked for me, don't really remember myself most of the steps anymore. Just looking at the log it seems to be having an issue detecting a matching device, so perhaps something else in HEX need to change?

      Delete
    2. Have you had any success on this yet? I too am trying to update the ST9600204SS to work with dell servers. I have a slew of them and would like to get them working.

      Delete
  12. I Was able to do this on a DELL r610 using Seagate brand ST3146356SS. I used the FRMW_WIN_R230278.EXE Firmware package and extracted it like instructed above. I edited the HS0F.FWH file a little differently to fit my model of drive. Instead of changing DELL.HS0FHS06 to DELL.HS0F0007 I changed it to DELL.HS0F0000 (I had tried DELL.HS0F0005 thinking it would match my version 0005 but it didn't work) and instead of changing 15301 to 00000 I changed 15303 to 00000(this matched my drive model). I ran the command "SASDUPIE.exe -u -s "%cd%\payload" -f -o update.xml -debug debug.log" in a command window from within the extracted FRMW_WIN_R230278 directory. this time it took a few minutes to complete and my version changed to HS0F. I then was able to update to version HS11 using the dell firmware upgrade program with no problems.

    I imagine the bootable usb method would work as well.

    Thanks Dmitri!!!!

    ReplyDelete
    Replies
    1. your disks are working normally, some error happened after the firmware change

      Delete
  13. I tried installing ES68 firmware on a ST3450857SS NA06 (Netapp) drive, I made the firmware changes but the update is not enabled, is there any procedure to run?

    ReplyDelete