A Couple of VMWare Issues

March 15, 2008

 

Sorry for the lack of posting this month; building a 200 person call center from scratch has been dominating my time. It is almost done though, but I still need to find time to post.

Today we discuss a couple VMware issues.

"Operation Failed Since Another Task Is In Progress"

I've seen this error a couple of times. Basically what happens is that a VM will show running but is actually frozen. Any attempts to force a VM shutdown or restart results in the error "Operation Failed Since Another Task Is In Progress". Same if attempted to Vmotion the machine.

Basically this turns out to be a snapshot issue. To fix this without rebooting the ESX server, we can just kill the VM process via command line. I took this tip from a VMware Communities post and cleaned it up a little. The post can be found here.


1. SSH into the ESX server that is currently running the affected VM (or you can use the console).
2. At the cmd prompt enter: cat /proc/vmware/vm/*/names

This lists the running VM's on the host server you are logged on to. Look for the vmid=##

vmid=1069 pid=-1 cfgFile="/vmfs/volumes/45.../server1/server1.vmx"
uuid="50..." displayName="server1"
vmid=1107 pid=-1 cfgFile="/vmfs/volumes/45.../server2/server2.vmx"
uuid="50..." displayName="server2"
vmid=1149 pid=-1 cfgFile="/vmfs/volumes/45.../server3/server3.vmx"
uuid="50..." displayName="server3"
vmid=1156 pid=-1 cfgFile="/vmfs/volumes/45.../server4/server4.vmx"
uuid="50..." displayName="server4"

3. At the cmd prompt enter: less -S /proc/vmware/vm/1149/cpu/status

It will now clear the console screen and show a bunch of numbers and stats. Hit the right arrow key until you see the section about group. Example:

group
vm.1058

With this ID number you can safely kill the VM without corrupting it.

4. At the cmd prompt enter: /usr/lib/vmware/bin/vmkload_app -k 9 1058

(Then number 1058 in the command is an example; your VM's group number goes here.)

5. If you see "Warning: Apr 20 16:22:22.710: Sending signal '9' to world 1058." this means your VM has been closed successfully. You can now start your VM back up and run it.

Unable to migrate due to "Remote Backing" issues with CD/DVD

When trying to VMotion two VM machines, I received the error: "Unable to migrate from VMESX2 to VMESX1: Virtual machine is configured to use a device that prevents migration: Device 'CD/DVD Drive 1' is a connected device with a remote backing.". Going into Edit Settings didn't help; all CD/DVD options where greyed out. I had recently set both systems to use the client device and to disconnect.

I was able to fix one of them; the VMware Tools was still waiting to be installed. I right-clicked on the VM and choose "End VMware tools install". That did the trick and it VMotioned fine.

The other system didn't have that option. I ssh'd into the esx host and from the command prompt ran service mgmt-vmware restart

After about two minutes (including a scary "disconnected" state in Virtual Center that lasted about a minute), it allowed me once again mess with my CD/DVD settings and I was once again able to Vmotion.

More on this issue in this VMWare Communities thread.

13 comments:

Anonymous said...

Hey! Thanks for that post, I've found it really helpful!

Anonymous said...

Finding this post caused my out of hours work to last 15 minutes, instead of all night. Lifesaver.

Thanks a lot.

Daniel

Anonymous said...

Awesome, thanks!

Nav said...

Awesome, saved our days work mate !!! even VMWare tech guys didnt have answers for these !!!

Anonymous said...

THANKS for sharing this!!!! :D

Erwin Blonk said...

Saved the day.

Patwij (Belgium) said...

Thanks, procedure worked nicely

Anonymous said...

finally a solution that works :) thanks a lot!

Anonymous said...

OMG... 5 hours of service outage remedied in less than 10 minutes.. You are the man.

Anonymous said...

Thanks man...
I was able to get my systems up and running in 30 mins with your instructions.
Kepp them coming.

AstroCrystal said...

I have this exact issue but the VM that is affected is not 'running' so it doesn't show up in the names list. Is there a way to get the VMID for a guest that isn't running?

Anonymous said...

Thank you. Awesome post. Easy to follow and the best part it works.

Anonymous said...

Just another satisfied user saying thanks very much!!