• Category Archives vSphere
  • ESXi Customizer and Windows 10 Fix

    ESXi-Customizer is a great tool to add vibs and custom drivers to a VMware ESXi installation. I had to do this for my lab for the Network interface drivers which I blogged about with instructions here.

    From the product page:

    ESXi-Customizer is a user-friendly script that automates the process of customizing the ESXi install-ISO with drivers that are not originally included. Unlike other scripts and manuals that are available for this purpose ESXi-Customizer runs entirely on Windows and does not require any knowledge of or access to Linux

    I recently went to use the ESXi Customizer as I am doing updates and refreshes of my lab (more on that later) and like many folks out there I have already taken the plunge into Windows 10 (actually have been running Win 10 Enterprise since Jan 2015).

    I got the following error message.

    Error: “!– FATAL ERROR: Unsupported Windows Version: 10.0. At least Windows XP is required!”

    Luckily there is an easy fix.  The ESXi-Customizer is a .cmd file which is easily editable so I did a search in Notepad++ (Great tool, go get it.)

    Resolution: Add a REM at the beginning of line 260 as follows. This will eliminate the Operating System check as it fails on Windows 10

    REM if “!WinVer!” LSS “5.1” call :earlyFatal Unsupported Windows Version: !WinVer!. At least Windows XP is required & exit /b 1

    Voila!! That’s it. It worked great in my lab (and according to folks from twitter)

  • Cyber Monday : Free VCP 5 Study Guide

    It has been a while since I have posted anything (expect more soon) but in the tradition of Cyber Monday where everyone is trying to sell things, I have decided to give something away for free. A copy of Brian Atkinson’s VCP 5 Study Guide. This is a solid book for studying for the VCP5 written by a fellow vExpert.

    Rules: simple, post a comment below and next Monday I will pick a random post and mail them a copy of the book, no strings attached.


  • VMware Fling : VIB Author – Custom ESXi installs

    VMware has announced another fling which I always enjoy trying. This fling will allow users to make a customized vSphere Installation Bundle (VIB). Previously this was only available to VMware partners such as Cisco, Dell etc to make customized installations.

    From the article

    There are a couple of use cases for creating custom VIBs.  For example, if you are using Auto Deploy and you need to add a custom firewall rule to your host, or you need to make a configuration change that can’t be made using Host Profiles.
    One word of caution however, the ability to create custom VIBs does come with some responsibility.  If you plan to create your own VIBs here are a few things to keep in mind:

    1. VIBs provided by VMware and trusted partners are digitally signed, these digital signatures ensure the integrity of the VIB.  Custom VIBs are not digitally signed.  Be careful when adding unsigned VIBs to you ESXi hosts as you have no way of vouching for the integrity of the software being installed.
    2. Before adding a custom VIB you will need to set your host’s acceptance level to “Community Supported”.   When running at the community supported acceptance level it’s important to understand that VMware support may ask you to remove any custom VIBs.   Here’s the formal disclaimer:

    IMPORTANT If you add a Community Supported VIB to an ESXi host, you must first change the host’s acceptance level to Community Supported. If you encounter problems with an ESXi host that is at the CommunitySupported acceptance level, VMware Support might ask you to remove the custom VIB, as outlined in the support policies:”

    If you are not familiar with VIBs I recommend you start with a quick review of this blog: http://blogs.vmware.com/esxi/2011/09/whats-in-a-vib.html
    With that, I know several folks have been chomping at the bit to create their own custom VIBs so I’ve attached a short tutorial that shows how to use the vibauthor tool to create a  VIB to add a custom firewall rule.

  • VMware vSphere 5.1 Known Issues : Updated 9/17/2012

    VMware released vSphere 5.1 at VMworld 2012 in San Francisco and with any new product release there are usually bumps on the road, compatibility issues and things that just don’t work. VMware vSphere 5.1 is no exception.

    I have been part of many beta tests for other companies as well as several vSphere editions going back to 3.x. Not every configuration and product can be tested. The beta process does catch a lot of issues. The day software goes live without any issues every time, a lot of people will be without work..

    I have decided to catalog a few of the known issues with the vSphere 5.1 release.

    Perhaps the larges issue is that VMware View and vSphere 5.1 are not compatible yet.

    Last Updated 09/30/2012

    vSphere 5.1 and VMware View (all versions of View) 

    VMware vSphere 5.1 is not currently supported with any versions of VMware View.

    Do not upgrade vSphere above the supported versions listed in the VMware View 5.1 Release Notes.

    vSphere 5.1 is in the process of being certified against VMware View. To be alerted when this article is updated, click Subscribe to Document on the link above in the actions box

    VMware vSphere 5.1 and Cisco UCS

    Read Chad Sakacc’s post for more details. vSphere 5.1 support on Cisco UCS, the VMEM needs a new vSphere 5.1 specific .vib.

    VMware vSphere 5.1 KB articles

    VMware complied several of the 5.1 KB articles in a recent blog post on VMware Support Insider blog

    More VMware vSphere 5.1 KB articles  ** Added 9/17/2012 **

    VMware has compiled more vSphere 5.1 KB articles that you may have missed. This one features a few of the Single Sign-On (SSO) issues that have been common for some of you

    vCenter Server Services hang on startup after upgrading to vCenter Server 5.1 

    From the VMware KB article: Follow the link for resolution

    • After upgrading to vCenter Server 5.1, the VMware VirtualCenter Server service fails to start
    • The VMware VirtualCenter Server service will not successfully stop or start.
    • The vpxd log files show the following:
      • The final log in the vpxd.log file shows:
        CoreDump: Writing minidump
    • You see a backtrace near the very bottom of the vpxd.log file which begins with the following:

    2012-09-13T11:53:46.802+02:00 [04468 info ‘vmmoVm’ opID=SWI-c5103928] [VmMo::SetComputeCompatibilityDirty] vm vm-5345 is marked dirty
    2012-09-13T11:53:46.786+02:00 [05112 warning ‘win32vpxdOsLayer_win32’ opID=SWI-56f04b9c] [VpxUnhandledException] Backtrace

    Increased Restart time with vCenter Server 5.1

    This issue only impacts the start/re-start time for vCenter Server. It does not affect the ongoing operations after vCenter Server has started. In fact, due to improvements to the vCenter Server database, the use of vSphere Web Client, the Inventory Service cache, customers will notice significant performance improvements in vCenter Server 5.1.

    VMware vSphere 5.1 and EMC VMAX arrays.

    Sphere 5.1 changes the behavior of VAAI Hardware Accelerated Locking (aka ATS) to no longer work with transient (sometimes on/sometimes off) ATS behavior, and older (i.e. non-current) versions of Enginuity will fail to create VMFS-5.

    VMware vSphere 5.1 and EMC PowerPath V/E 

              **** Updated 09/26/2012 Resolved *** – Fix is out.

    PowerPath/VE customers, hold off vSphere 5.1 upgrades (GA was yesterday).   Hotfix P02 from EMC is in days, and so is the expected VMware fix, follow the above link for more details on Chad Sakac’s blog post

    VMware vSphere 5.1 and Synology DSM 4.1   ** Blog updated on 9/18/2012 **

    Kendrick Coleman found an issue with the most recent release of DSM 4.1 and vSphere 5.1

    It’s possible to mount both NFS and iSCSI datastores, but when you try to build or power on a VM, everything begins to halt when it needs to write to disk. If everything runs in memory (like mounting an ISO and beginning an OS installation), but when it comes to install the OS to disk, it begins to crawl. I checked the vmkernel log and here is what is shown. You can see that after I power on the VM, it gets binded to a port but when it tries to read from disk, there are all sorts of errors 

    Update Manager 5.1 reports the compliance status as Incompatible when scanning or remediating ESXi 5.x hosts that belong to an HA cluster 

    I personally ran into this issue when upgrading my lab – see Update Manager 5.1 Release Notes

    CA Signed SSL Certificates may cause trouble with the upgrade process of vCenter.

    From Michael Webster’s blog at http://longwhiteclouds.com/

    I have heard unconfirmed reports of difficulties with the upgrade process of vCenter particularly with registering Inventory Service and SSO with vCenter when using CA Signed SSL Certificates. As I’m using CA Signed Certificates in my lab environment I will update this article when I have completed my upgrade.

    Troubleshooting SSL certificate updates and Single Sign On (2033240)
    vCenter Single Sign On installer reports: Error 29155. Identity source discovery error (2034374)

    ESXi cannot distinguish between EagerZeroedThick & LazyZeroedThick
    From Cormac Hogan’s Blog

    ESXi cannot distinguish between thick provision lazy zeroed and thick provision eager zeroed virtual disks on NFS datastores with Hardware Acceleration support

     When you use NFS datastores that support Hardware Acceleration, the vSphere Client allows you to create virtual disks in Thick Provision Lazy Zeroed (zeroedthick) or Thick Provision Eager Zeroed (eagerzeroedthick) format. However, when you check the disk type on the Virtual Machine Properties dialog box, the Disk Provisioning section always shows Thick Provision Eager Zeroed as the disk format no matter which format you selected during the disk creation. ESXi does not distinguish between lazy zeroed and eager zeroed virtual disks on NFS datastores.

    VMware vSphere 5.1 iPXE problems with AutoDeploy

     If you are having issues with IPXE and AutoDeploy please read Gabe’s Virtual World

    VMware Support Insider: ALERT: Full disk on vShield Edge 5.1.x fails with error: VIX_E_DISK_FULL ERROR

    Added 09/30/2012

    VMware has become aware of an issue whereby vCloud Networking and Security 5.1 release can go into an Edge disk full state approximately 14 days after the first edge is deployed.

    Symptoms include:

    • In vCloud Director, attempting a reconfig fails with this error:VIX_E_DISK_FULL
    • In vCloud Director, when looking at Edge Gateways, you receive this error:Edge VM backing the edge gateway is unreachable

    For further updates and more information on this alert, refer to KB article:
    Full disk on vShield Edge 5.1.x fails with error: VIX_E_DISK_FULL ERROR (2035939)

    Links for more information
    vSphere 5.1 Important Upgrade Considerations by Michael Webster VCDX #66 

    As with any upgrade please carefully consider especially before implementing in production. Reach out to your partners and trusted advisers for guidance so they can evaluate your environment to minimize issues.

  • Home Lab – a great resource

    I have been looking at building a lab for quite some time, years actually.

    Well I finally pulled the trigger, which I couldn’t do without the support of my company Varrow and my wife. Check out Jason Nash’s blog post on “In Support of the Home Lab” on how Varrow really takes it to the next level for supporting home labs and I think other companies should step up and help their employees too as the lab is a win/win situation for all parties involved.

    Read more for details on equipment and some of the trials and tribulations I have gone through thus far.

    There are many blog posts from folks in the community on building a home lab and what equipment they chose. I will include some of those articles at the end to give you more ideas on what to use in your lab.

    Here is the equipment that I chose, all purchased from Newegg.

    Case LIAN LI PC-V351B Black Aluminum MicroATX Desktop Computer
    Motherboard SUPERMICRO MBD-X9SCL-F-O LGA 1155 Intel C202 Micro ATX
    Power Supply Rosewill Green Series RG630-S12 630W Continuous @40°C,80
    CPU Intel Xeon E3-1220 Sandy Bridge 3.1GHz LGA 1155 80W
    Memory Kingston 8GB 240-Pin DDR3 SDRAM DDR3 1333 ECC Unbuffered
    SSD – Internal Crucial V4 CT032V4SSD2BAA 2.5″ 32GB SATA II MLC Internal
    NAS Synology DS212 Diskless System DiskStation – Feature-rich 2-bay
    Storage Seagate Barracuda ST3000DM001 3TB 7200 RPM SATA 6.0Gb/s

    All total this configuration cost me $2139.83 which again I couldn’t do without my employers generous lab policies after all I have four boys to feed and a wife who I want to keep happy.

    Notes on why I chose each piece for my lab as I put a lot of thought and planning into it. I had limited space and certain things I wanted to do and a set budget I had to get mine done in. You could choose other parts and save some money as well.

    • Case  – Chose the case due to size mainly and aesthetics, had to pass Wife Acceptance Factor
    • Motherboard – I wanted the Integrated IPMI 2.0 with KVM and Dedicated LAN for remote management
    • Power Supply – The power supply was on sale and got solid reviews for being quiet improving the WAF (wife acceptance factor)
    • CPU – Processor supports VT-D and was the best deal for my money
    • Memory – Best price Unbuffered ECC RAM i could find *Motherboard Requires Unbuffered ECC
    • SSD Internal – Wanted to use the Swap to Host Cache aka Swap to SSD Features in vSphere 5
    • NAS – Synology – accept no substitute -AWESOME, feature packed cant be beat, supports iSCSI and NFS and more. See Jason Nash’s review on the Synology 212+.
      • Only Regret – Wish I bought the DS412+ it supports VAAI with latest code and holds more drives
    • Storage – Most capacity and speed for the price I could get

    Watch the sales on Newegg, they constantly have things going out. If you are smart shrewd and have time on your hands, you could easily cut the cost down significantly.

    On to the build process

    So everything arrived from Newegg and i was like a kid in a candy store. The wife graciously allowed me to begin putting the pieces together as she wanted the boxes to disappear. I had a few issues such as a dead hard drive, the wrong ram was sent but most of that is being taken care of. I do have to say its a pretty whisper quiet set up. Even in the same room, I can’t hear the equipment minus a Cisco Switch which I have…

    The unboxed equipment waiting for me at home
    The Final Product with glowing lights.
    A few of my trials and tribulations

    I didn’t purchase a CD-Rom for the LAB so my plan was to install from a USB key. I found a nice utility (LinuxLive USB Creator) that you can use to create a bootable USB from any .iso file. I downloaded vSphere 5.0

    After I created my bootable USB with ESXi 5.0 Update 1 I thought I was in the clear and ready to begin installation. After I installed ESXi I ran into a bit of a snag, the NICs on my Host are not supported…

    I encountered this error “No compatible network adapter found. Please consult the product’s Hardware Compatibility Guide (HCG) for a list of supported adapters.

    There are two LAN controllers on the motherboard, neither are supported by ESXi 5.x

    • LAN Chipset
      Intel 82579LM
    • Second LAN Chipset
      Intel 82574L Duel NICs

    After some searching and afraid I was going to have to spend more money or write my own driver (been a while) I found another enterprising soul had written a driver that should be compatible with my board. VMware has a KB article on how to installing ASYNC drivers on 5.x here but there is an app for that, the ESXi customizer. The drivers supported the Intel 82574L Chipset so I currently have two Gigabit NICs. The primary NIC is non functional. I will at some point buy additional NIC cards to support more ports.

    Driver Author’s Post 
    Driver Download
    ESXi Customizer

    Download the driver above and the ESXi Customizer. It will create a custom ISO in the working directory that you can then use the LinuxLive USB Creator above to create your bootable ISO that supports the motherboard NICs.

    So after all of this I now have two ESXi hosts built at home and I am working on building the vCenter, Domain controller and other VMs soon.

    My plan is to install and do nested hypervisors for testing and script development against multiple platforms. As I make progress I will be giving LAB updates here and new scripts to share with the community.

    Nested Xenserver

    Nested Hyper-V

    Software I plan on developing for and testing in my LAB in no particular order and by no means complete just what I am thinking off the top of my head.

    • Citrix XenApp Plat using Citrix PVS, Edgesight, Single Sign On, Smart Auditor
    • Citrix XenDesktop Plat using MCS and Citrix PVS
    • Citrix Netscaler Access Gateway
    • Citrix APP-DNA
    • Citrix VDI in a Box
    • VMware View
    • VMware vCloud Director
    • VMware Horizon Suite
    • Appsense
    • Veeam
    • Windows Server 2012
    • App-V
    • Thin Clients
    • Mobile Devices and management 
    • Certification Testing and Guides

    All in all I can’t recommend building a lab enough, I think in this business building a home lab whether it be virtual using the Autolab or building a full home lab with multiple hosts or even a single host is a requirement for any professional. Every time I build anything I learn something even if its just that the Aluminum case edges are sharp 🙂 I believe it is the best way to keep your skills sharp, test new products and test yourself against them.

    Home Lab Links by the community
    Jase McCarty
    Hersey Cartwright
    Jason Boche

  • RVTools 3.3 Released

    RVTools is one of my favorite VMware community tools that every VMware admin should be using. Today version 3.3 has been released. I have been using this tool for many years and it has saved me from disaster many times.

    RVTools is a windows .NET 2.0 application which uses the VI SDK to display information about your virtual machines and ESX hosts. Interacting with VirtualCenter 2.5, ESX Server 3.5, ESX Server 3i, ESX Server 4i, VirtualCenter 4.0, ESX Server 4.0, VirtualCenter 4.1, ESX Server 4.1, VirtualCenter 5.0, VirtualCenter Appliance or ESX Server 5 RVTools is able to list information about VMs, CPU, Memory, Disks, Partitions, Network, Floppy drives, CD drives, Snapshots, VMware tools, ESX hosts, HBAs, Nics, Switches, Ports, Distributed Switches, Distributed Ports, Service consoles, VM Kernels, Datastores and health checks. With RVTools you can disconnect the cd-rom or floppy drives from the virtual machines and RVTools is able to update the VMware Tools installed inside each virtual machine to the latest version.

    This Application Supports:

    •  ESX Server 3.5
    • VirtualCenter 2.5
    • ESX Server 3i
    • ESX Server 4i
    • VirtualCenter 4.0
    • ESX 4.0
    • VirtualCenter 4.1
    • ESX 4.1
    • VirtualCenter 5.0
    • VirtualCenter Appliance
    • ESX 5

    Changes in 3.3

    • GetWebResponse timeout value changed from 5 minutes to 10 minutes (for very big environments)
    • New tabpage with HBA information
    • On vDatastore tab the definition of the Provisioned MB and In Use MB columnswas confusing! This is changed now.
    • RVToolsSendMail accepts now multiple recipients (semicolon is used as separator)
    • Folder information of VMs and Templates are now visible on vInfo tabpage
    • Bugfix: data in comboboxes on filter form are now sorted
    • Bugfix: Problem with api version 2.5.0 solved
    • Bugfix: Improved exception handling on vCPU tab.
    • Bugfix: Improved exception handling on vDatastore tab.

    Links –
    Offical Site
    Download RVTools