Fix for Cron Failing on VMware vCenter Server Appliance (VCSA) 6.5

When trying to enable scheduled jobs via cron on VMware VCSA 6.5 I kept seeing the errors below, and my job would not run.

2017-04-19T09:56:01.996673-04:00 VCSA crond[104661]: PAM _pam_load_conf_file: unable to open config for password-auth
2017-04-19T09:56:01.996797-04:00 VCSA crond[104661]: PAM _pam_load_conf_file: unable to open config for password-auth
2017-04-19T09:56:01.996907-04:00 VCSA crond[104661]: PAM _pam_load_conf_file: unable to open config for password-auth
2017-04-19T09:56:01.997010-04:00 VCSA crond[104661]: (root) PAM ERROR (Permission denied)
2017-04-19T09:56:01.997116-04:00 VCSA crond[104661]: (root) FAILED to authorize user with PAM (Permission denied)

The contents of /etc/pam.d/crond had 3 references to “password-auth”, however there was no file in /etc/pam.d called “password-auth”.  I changed “password-auth” to “system-auth” in /etc/pam.d/crond, as seen below, and everything worked.

account required pam_access.so
account include system-auth
session required pam_loginuid.so
session include system-auth
auth include system-auth

VMware vCenter Storage View is blank/not updating

We upgraded our VMware infrastructure to v5 back around the time it was released late last summer. Since then I’ve been having an issue where the Storage View feature of vCenter would not load. I would receive a mostly blank screen that has a link to update the Storage View. Clicking that link had 0 effect. I finally started messing around trying to troubleshoot a few weeks ago (after nearly 6 months without Storage View). Something I did resolved the issue and it seemed to be working since.

Today I loaded my VI Client up and clicked the Storage View tab to check on something. To my displeasure I realized SV was broken again. This time, I finally determined why – the server vCenter is installed on had a port conflict with another process, the port in question being 8080, which was also being used by IIS for some other service I had installed at some point in the past (certainly NOT at the time that SV broke originally). I changed the port that the IIS instance was running on, restarted the VMware services, and voila, SV worked again.

Like I said above, the IIS instance was running prior to upgrading to vSphere 5, so I’m not sure exactly what caused the conflict. I don’t know if vCenter 5 installs a new service to port 8080 by default or if I, in my infinite wisdom, set the port to use 8080. I know I set IIS to 8080 at one point to AVOID a port conflict with vCenter, however, I wasn’t aware that vCenter was running any web services on ports other than 80 and 443. Long story short, make sure that the vCenter Management Webservices HTTP service isn’t conflicting with anything else (according to VMware 8080 is the default port for the aforementioned service).

VMware ESX/ESXi 4–Windows Server 2008 R2 Mouse Choppy <Solved>

I had made a previous post a while back about the mouse being choppy while using the VMware console with Server 2008.  I stated in that post that as of that writing, there was no fix for 2008 R2.  There appears to be a fix now, however.  The server I’m running is ESX 4.1.0-348481 (which is the RTM of ESX 4.1 Update 1).  To get the mouse choppiness to stop, open up Device Manager and select the display adapter.  Update the driver and manually choose the location of the driver (which is located in C:\Program Files\Common Files\VMware\Drivers\video).  After selecting that location and clicking “Next”, it automatically found the proper driver and installed it, and after a reboot, the mouse choppiness was fixed.  I had already enabled hardware acceleration for the adapter, so if you update the driver and are still having issues, ensure you’ve enabled hardware acceleration.  I’m also running the latest version of VMware Tools on this machine, so if this directory doesn’t exist, or this doesn’t work for you, make sure VMware Tools are up to date.

VMware ESX/ESXi–Find All Snapshots

One of the best things about virtualization is the ability to take snapshots of virtual machines while running or while powered off.  This saves countless hours spent taking backups of VM’s prior to upgrades or critical reconfigurations.  If the change causes problems, simply revert  back to the snapshot that you took beforehand.

It is best practice not to keep snapshots for an extended period of time, as delta (change) files related to the snapshot begin to grow and can become unstable at some point (I’ve had instances where my VMware server glitched out and ate up all storage space in my datastore by creating snapshot files).  This being the case, it is best to purge unneeded snapshots as soon as you’re certain you don’t need them anymore.  However, what if you have dozens or hundreds of VMs and don’t remember which ones have snapshots?  You could click on each VM and see if there are any snapshots associated with it, or you can use this technique I just discovered this morning to view all snapshots in  your datacenter.  This is only applicable to vSphere 4.0+.

Open the vSphere Client and browse to Home –> Inventory –> Hosts and Clusters (in the top navigation pane).  Select the name of your datacenter in the left-hand pane.  Click the “Storage Views” tab.  For View, select “Reports”. Ensure that “Show all Virtual Machines” is selected from the dropdown menu directly below the “Reports” button.  There should be a column that says “Snapshot Space”.  If not, right-click the title columns and select “Snapshot Space” to add that column.  Then click the “Snapshot Space” column to sort it by size.  You can ignore VMs that have a Snapshot Space of a few bytes or a few KBytes.  Look out for ones that have a few GB of Snapshot Space.  These are the VMs you need to remove snapshots from.  Picture below has key areas circled.

VMware ESX/ESXi–Choppy Mouse Movement

I use VMware ESX/ESXi on a daily basis.  While I use LogMeIn as my remote management software for all computers in my environment, the vSphere Client also provides a console view to access my VMs.  Sometimes it’s more convenient to use the vSphere console, when performing tasks like modifying settings on the fly or inserting ISO files.

The VMware Tools application that should be installed on all VMs provides a number of enhancements, including support to allow for smooth mouse movement.  However, I found that my Windows Server 2008 VMs were having very choppy mouse movements, even with the VMware Tools installed.  The solution for this is to set the hardware acceleration for the video adapter of the VM to “Full”.

To do this, open the Display Resolution settings by right-clicking on the desktop and selecting “Personalize”.  Then select “Display Settings”.   In the Display Settings window, click “Advanced Settings”, click the “Troubleshoot” tab, click “Change Settings”, then move the Hardware Acceleration slider all the way to the right, then click “Ok”.  You will be prompted to reboot.  After you reboot, your mouse movements should nice and smooth.

This only works on 2008, not 2008 R2.  Feel free to try this on previous versions of Windows if you’re having mouse issues, although this setting is usually in place by default, but it’s a confirmed bug with 2008 R2.

VMware Server 2.x – a nightmare

For the past year and a half we’ve been running one VMware product or another.  We started with VMware Workstation and Server, and while Workstation proved to be (and continues to be) an invaluable tool, Server just wasn’t cutting it.  I had used Server 1.x at home to host a few virtual servers without any issues for the better part of a year.  Server 2.x, however, which we used at the office, was a nightmare.  To those not too familiar with VMware’s products, Server is a Windows (or Linux) application hypervisor.  It is just like Microsoft Virtual Server.  You install Windows on the hardware, install VMware Server, then install virtual machines on top of VMware Server.  For a server with enough horsepower and that is not running any critical applications, installing VMware Server on top seems like a great way to break into the virtualization world.

Not so much, we found, with VMware Server 2.x.  The way virtualization works is that hard drives are stored as a single container file, 1 per virtual hard drive (from here on out referred to as “VMDK”).  Over the course of a number of months when we ran Server, we had a number of instances, on both machines that were running Server, where the VMDK files would become corrupt (multiple files at a time) and render the VM useless.  Not only that, but VMware Server would not start the services necessary to run or manager the VMs when it detected these corrupt files.  This was a bug in the software; there were no problems with the physical hard drives.  After dozens of hours of trial and error, I came up with a method of recovering the corrupt VMDK files and rescuing the contents to a new, useable VMDK.  After a while, we finally had enough, and moved on to ESXi.