bitsmarts: 2013

April 8, 2013

The trust relationship between this workstation and the primary domain failed

Had this error happen with a Microsoft Windows 2008 R2 server running in a VM.
When logging in to Windows, if using any domain accounts, getting the error

“The trust relationship between this workstation and the primary domain failed"

The obvious fix is to rejoin the domain but a much simpler and probably less invasive method is to run the netdom command

Login with a local machine admin account and run:

netdom.exe resetpwd /s:<server> /ud:<user> /pd:*

<server> = a domain controller in the joined domain
<user> = DOMAIN\User format with rights to change the computer password. I used domain admin.

More info at
http://implbits.com/About/Blog/tabid/78/post/don-t-rejoin-to-fix-the-trust-relationship-between-this-workstation-and-the-primary-domain-failed/Default.aspx

March 5, 2013

ESX 4.0 upgrade to ESXi 5.1 with 3ware 9650SE

Intro

This is a straight up upgrade of an ESX 4.0 installation to ESXi 5.1.

The hardware is Intel Xeon CPUs however the biggest obstacle was the RAID controller LSI/AMCC 3ware 9650SE which was not detected when installing with the VMware default ISO.

The RAID1 was also where the ESX 4 was installed, so it was not even possible to perform the upgrade without the extra steps.

Overview

Download the VMWare driver for the 3ware 9650SE
Integrate the driver into a custom boot ISO image
Perform the upgrade using a CD on site
Few observations and notes

--------------------------------------------------------------------------

1. Download the VMWare driver for the 3ware 9650SE

Search the web for 'VMware Compatibility Guide'. At the time of writing, the URL is http://partnerweb.vmware.com/
Change

'What you are looking for' to 'IO Devices'
select ESXi 5.1
and keyword '9650'

In the results click on the '3ware 9650SE' link. Brand name is under 'AMCC'
The footnotes refer to

'3w-9xxx version 2.27.08.036' and a link
http://www.vmware.com/download/vsphere/drivers_tools.html

Follow the link, and either

search for '3w-9xxx version 2.27.08.036' in 'All Downloads'
or follow the links

VMware vSphere ->
Drivers & Tools tab ->
Driver CDs ->
look for '3ware' or '2.27.08.036'

Final link is at VMware ESXi 5.0 Driver CD for LSI 3ware 96xx RAID Controllers
File LSI_2.27.08.036vm50-637081.zip (110,254 bytes)

2. Integrate the driver into a custom boot ISO image

Search the web for 'ESXi-Customizer'. At the time of writing, the URL is

http://www.v-front.de/p/esxi-customizer.html ver. 2.7

Install and run the ESXi-Customizer.cmd
Unzip the driver file, and feed the ISO image and the driver into the ESXi-Customizer
Files are:

VMware-VMvisor-Installer-5.1.0-799733.x86_64.iso
scsi-3w-9xxx-2.27.08.036-1OEM.500.0.0.472560.x86_64.vib

Once the ISO image is built, burn it to a CD or DVD, rewritable also works.

3. Perform the upgrade using a CD on site

This is a straight forward procedure which should not take more than 30min
Schedule a maintenance window, and reboot with the CD in the host
The RAID will now be detected as a storage device and you can perform the 'force upgrade'
After the reboot, make sure you have your new licence ready because it will be in a 60 day trial mode

4. Few observations and notes

The 4.0 was actually updated (not upgraded) using the Update Manager first. There was a few hundred patches and updates, however I don't believe this was necessary prior to the upgrade to 5.1
The Update Manager could most likely use the custom ISO as a new upgrade baseline, however this was not tested.
First attempt was the 5.1 default ISO which was used as an upgrade baseline in the Update Manager, however this failed and caused the host to go in an infinite reboot loop. There is a boot menu that was added that allows either boot from 4.0 or to perform the 5.1 upgrade. Not sure how to edit this menu and remove the 5.1 upgrade as default option which kept failing

March 4, 2013

Update VMWare Tools on Cisco CallManager 7.1 to ESXi 5.1

Intro

We have CUCM 7 running on ESXi 4.0 that was upgraded to 5.1. VMWare tools need to be installed or updated on the CallManager

Overview of the steps

Log in to the CUCM shell via "Remote Support" account
Initiate the VMWare tools install
Bonus: Troubleshooting an error 'No module mptscsi found'

1. CUCM Remote Support login

Download the CentOS install disk. Tested working is

CentOS-5.5-i386-bin-1of7.iso (653,910,016 bytes)

admin:utils remote_account enable
admin:utils remote_account create [username] [num of days active]

Mount the CentOS ISO to the VM and reboot the CM

admin:utils system restart

Boot to the CentOS CD, press 'esc' key during boot to get to the BIOS boot manager
Select the 'recovery disk' option
Mount the root partition '/' as there may be '/partB' as well
Once you get to shell prompt, check these files

#lsattr /etc/passwd /etc/group /etc/shadow /etc/gshadow

In my case they were all --------, but if there's an immutable flag 'i' you will have to take it out

#chattr -i /etc/passwd /etc/shadow /etc/group /etc/gshadow

Change the password for the [username] used above in step 2

#passwd [username]

Reboot and unmount the CD from the VM
Log in to the console using the [username] and password you reset in step above

2. VMWare Tools install

In the vSphere client, initiate the vmware tools installation by going to Guest-> 'Install/Upgrade VM Tools'. Don't select the automatic install.
Go to the console where you logged in with [username] and check that you have a folder /media/cdrom
In my case, there was no /media/cdrom. Because of an upgrade it was located in /partB/media/cdrom however the /etc/fstab was still pointing to /media/cdrom
I had to create folder /media/cdrom before mounting the CD, then just issue

# mount /media/cdrom

Copy the tools tar.gz from the CD to your home folder then untar and run the install script

# cp /media/cdrom/VMwareTools-9.0.1-913578.tar.gz ~
# cd ~
# tar zxvf VMwareTools-9.0.1-913578.tar.gz
# cd vmware-tools-distrib/
# ./vmware-install.pl

Take all the defaults the script offers. If you don't have errors, this should complete fine, in which case just reboot the server and enjoy the job well done.

3. BONUS. Errors!

During the install, just at the end it fails with an error message

'No module mptscsi found for kernel x.x.x.x'

There's a lovely KB article on VMWare site at

Upgrading VMware Tools fails with the error: No module mptscsi found for kernel x.x.xx-xx, aborting

Even though the kernel versions don't match, following the instructions to comment out 'mptscsi' from /etc/modprobe.conf worked. In my case I had to comment out two entries,

/etc/modprobe.conf :
...
#alias scsi_hostadapter1 mptscsi
...
#alias scsi_hostadapter5 mptscsih
...
~

Re-running the 'vmware-install.pl' script completed successfully this time, and I uncommented those entries before the final reboot.

January 19, 2013

Enable VNC in VMware ESXi 5.1

Steps

create custom firewall rule file
refresh rules
make sure the changes are persistent across reboots
NOTE: these changes may not be persistent across upgrades or updates.
Enable VNC on the target VM

Procedure

Custom Firewall rule file

Service ID

First, find the service ID to use for your custom firewall rule.

Example:

/etc/vmware/firewall # cat service.xml
...
<service id='0032'>
<id>vprobeServer</id>
<rule id='0000'>
<direction>inbound</direction>
<protocol>tcp</protocol>
<porttype>dst</porttype>
<port>57007</port>
</rule>
<enabled>false</enabled>
<required>false</required>
</service>

</ConfigRoot>

/etc/vmware/firewall #

We are going to use 0033

Custom File

Create a file on your computer called enablevnc.xml


<ConfigRoot>


<service id="0033">
<id>VNC</id>
<rule id='0000'>
<direction>outbound</direction>
<protocol>tcp</protocol>
<porttype>dst</porttype>
<port>
<begin>5800</begin>
<end>5999</end>
</port>
</rule>
<rule id='0001'>
<direction>inbound</direction>
<protocol>tcp</protocol>
<porttype>dst</porttype>
<port>
<begin>5800</begin>
<end>5999</end>
</port>
</rule>
<enabled>true</enabled>
<required>false</required>
</service>

</ConfigRoot>

Copy the file to a datastore on your VM

in this example copied to

/vmfs/volumes/500G2/enablevnc.xml

You can also edit in place in SSH and just copy/paste.

Copy the file from the datastore to the firewall folder

/etc/vmware/firewall # cp /vmfs/volumes/500G2/enablevnc.xml /etc/vmware/firewall/

Refresh firewall rules

/etc/vmware/firewall # esxcli network firewall refresh

Confirm the changes

/etc/vmware/firewall # esxcli network firewall ruleset list

...

vSPC false

remoteSerialPort false

vprobeServer false

VNC true

Make changes peristent

Edit local.sh

ESXi 5.1 made changes so you have to edit /etc/rc.local.d/local.sh now.

/etc/vmware/firewall # vi /etc/rc.local.d/local.sh

#!/bin/sh

# local configuration options

# Note: modify at your own risk! If you do/use anything in this

# script that is not part of a stable API (relying on files to be in

# specific places, specific tools, specific output, etc) there is a

# possibility you will end up with a broken system after patching or

# upgrading. Changes are not supported unless under direction of

# VMware support.

cp /vmfs/volumes/500G2/enablevnc.xml /etc/vmware/firewall/

exit 0

/etc/vmware/firewall #

Run auto-backup.sh

/etc/vmware/firewall # /sbin/auto-backup.sh

Files /etc/vmware/dvsdata.db and /tmp/auto-backup.61769//etc/vmware/dvsdata.db differ

Saving current state in /bootbank

Clock updated.

Time: 03:54:44 Date: 01/20/2013 UTC

/etc/vmware/firewall #

Enable VNC on target VM

Make sure your VM is shut down

edit the .vmx file manually over SSH or change the 'Configuration Parameters' in vSphere

These are the two config parameters to add.

There is no password configured in this example.

remotedisplay.vnc.enabled = "true"

remotedisplay.vnc.port = "5900"

Connect

To connect to the desktop of the target VM, use a free VNC client, IP address of your ESXi server and port 5900