Quantcast
Channel: VMware Communities: Message List
Viewing all articles
Browse latest Browse all 258290

Re: ESXi 6.5 stuck on Starting service vmtoolsd

$
0
0

This problem could be potentially caused by VT-d enabled in the bios with Adaptec controllers.  Try disabling this first (though, I believe this issue was fixed in newer driver versions, it wouldn't hurt to rule this out if you don't require hardware pass-though).

 

If you are still getting the same issue with only the driver installed (not vmware-esx-provider-arcconf.vib or vmware-esx-provider-arc-cim-provider.vib as they are not required for accessing the data, only management of the controller / array), then you could potentially be having a more serious problem with the array.

 

Controller hangs / continuously scrolling "Host adapter abort requests" can also occur if the array contains a "badly behaving drive" when ESXi attempts to load the data store.  By "badly behaving drive", I mean specifically the drive is not responding to commands from the controller and the controller is (for whatever reason) unable to drop the drive from the array.

 

If you are using a 5-series or 6-series controller without a SAS expander, you can enable hard drive activity lights from within the controller's bios settings during boot up of the server (CTRL-A).  It might be possible to identify the "badly behaving drive" by observing the hard drive activity lights (I believe they are green LEDs on the controller).  The bad drive will sometimes illuminate the activity light continuously while others with flash as the drives are accessed.

 

If you have a 7-series or newer controller, I believe Adaptec had removed the activity LEDs from these controllers, so you would not be able to identify a drive in this fashion.  You would need some sort of HDD backplane which would illuminate a drive activity LED when the drive is being accessed by the controller.

 

If this is the case *AND* if you have some remaining redundancy within your array (eg a RAID-6 array with one failed disk and *ALL* remaining disks are working flawlessly with no undiscovered bad blocks), you could try disconnecting the "badly behaving" drive.  *** USE CAUTION *** as if this does not solve the problem, you cannot simply re-add the disk to restore the previously level of redundancy without performing a rebuild.  If you have multiple failed/failing disks, you could experience a total loss of your array data by doing this.  Proceed at your own risk.

 

If the above is not the case, you could try booting the server using an Adaptec supported Linux distro or Windows and see if the OS reports similar abort commands.  Linux might provide more detailed information as to what is going on with the controller/array.  If you are successful in booting Linux or Windows without having the system lock up, you could attempt to download the controller's firmware logs by creating a "support archive" and reviewing those logs.  The "support archive" logs often have more detail as to what his happening within the controller/array.  ***USE CAUTION*** to NOT write any data to the array while in Windows or Linux as it could permanently damage the VMFS filesystem. Proceed at your own risk.

 

*** Additional Caution ***

If you have a 5-series or 6-series controller in RAID-6 with >2TB disks, DO NOT attempt to rebuild the array.  There is an undocumented firmware bug in these controllers that *WILL* corrupt your data if you attempt a RAID-6 rebuild with >2TB disks while there is background write activity happening on the array (such as VMs running).  The issue may also apply to RAID-5 arrays, but has not been confirmed as far as I'm aware.  The same applies to older firmware versions of 7-series and 8-series controllers, though newer firmware resolved this issue for these controllers.

 

*** Additional Caution #2 ***

Don't change the firmware of the controller while the array is in a degraded state.  It is recommended to first get the array up to "Optimal" state before changing firmware levels on the controller.


Viewing all articles
Browse latest Browse all 258290

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>