How To: Upgrade vCloud Director in Detail – Part 1

Upgrading the vShield Manager

Upgrading the vShield Manager utilizes the UPGRADE package file. Do NOT deploy a new vShield Manager or all the information in the local MySQL database containing deployed vShield Edge devices will be lost.

WARNING!!! – Do NOT attempt to redeploy the latest version of the vShield Edge Appliance. Removing the existing vShield Manager Appliance will break all connections, and management to any deployed vShield Edge devices resulting in errors. Once a vShield Manager is deployed you should upgrade it in place to the latest supported versions to ensure the local database remains intact. It may be possible to restore a configuration to a newly deployed vShield Manager using the FTP backup taken, but that has not been tested for this document.

Back Out Plan for vShield Edge Devices

The feeling among many folks is that once you have committed to upgrading the deployed vShield Edge devices you are past the point of no return. Trying to back out at this point could result in data loss and possible corruption of vShield Edge networks and devices. Again, once users are back on the system there is definitely no going back as new objects will all be created using the new products.

However there are two options for providing a method of rollback it is up to each individual to decide the best method

  • Snapshot the virtual machine – This will require FT is disabled if you had it enabled
  • FTP Backup – This is available in the Web UI

Procedures

  • Snapshot the virtual machine OR
  • Back up the local database using the Web UI FTP option under “Settings and Reports”

Figure 1 – vShield Manager Backup Option

  • Upload the upgrade file using the “Updates” Section with “Upload Settings”

Figure 2 – vShield Manager Update Screen

  • Once the file is uploaded you will see be able to select “Install”

Figure 3 – vShield Manager Update Install Screen

Figure 4 – vShield Manager Update Steps Screen

  • The vShield Manager will go through reboots and when it comes back up it should show version 5.0 as the installed release. It will take a few moments for all the services to start before you can connect to the UI via the URL.

Figure 5 – vShield Manager Updated Build Screen

  • vShield Manager 5.0 uses a security model different from the previous version. After the install is done you can still log in with the “Admin” account and password you used before, but you will see it is no longer connected to vCenter Server.
  • You must wait 5-10 minutes per the release notes for vCloud Director to reconnect and populate the vCenter connection information.
    • This is NOT something you can change and has been identified as a possible feature request to allow vShield Manager and vCloud Director to use separate accounts. At this time vShield Manager will use the same credentials that are configured on the General Tab in the vCloud Director vCenter server general settings.
    • This is currently the expected behavior and has been verified by the vShield engineering team.
  • Lastly, you MUST add the vShield 5.0 licenses into vCenter Server as they are not the same as the old version. Your upgraded vShield will show as Evaluation Mode until you do so.

Figure 6 – vShield Licensing Updates

  • Validate everything is working by deploying a new NAT routed network to ensure a new vShield Edge is properly deployed before attempting to upgrade existing Devices.
  • Here you must decide on a few things:
    • Upgrading all the deployed vShield Edge devices on Org Networks
      • If you decide to do this keep users off the system and proceed to 3.6.2
    • Upgrading all the deployed vShield Edge Devices on vApp Networks.
      • If you decide to do this as an administrator keep users off the system and proceed to 3.6.2
      • You can elect to allow vApp owners to do this on their own.

Validation Steps to Perform After vShield Manager is Upgraded

Do not perform this step until you have seen that the update from vCloud Director has populated the vCenter connection in vShield Manager. If you do not wait long enough you will get errors when trying to deploy or reset networks.

The following two simple steps should be performed to ensure that functions are still working before moving to the next major step. These are basic but will ensure that there is a baseline of functionality as you move forward.

  1. Deploy a new vApp
  2. Create a NAT routed network

Rolling Back vShield Manager

If you decide at this point to roll back the installation you can revert/delete the snapshot. You should not need to restore the database from FTP backup. Again, if you continue past this point you are essentially past the point of no return.

Upgrade Deployed vShield Edge Devices

Be sure to wait at least 5-10 minutes after upgrading the vShield Manager for it to report back to vCloud Director that it is updated. There is a documented issue in the release notes here:

http://www.vmware.com/support/vcd/doc/rel_notes_vcloud_director_15.html#installissues

System administrators will need to upgrade any Org Network-based vShield Edges. Org Admins can upgrade any vApp Network vShield Edges, OR the System admins can upgrade them. It is recommended that at least for the Org Networks, users remain off the system and the load balancer remain redirected to the maintenance page until the process is completed.

  • Examine the version of a vShield Edge in vCenter and you will see the old build number

Figure 7 – vShield Edge Versions

  • Upgrading the vShield Edges that are deployed is very simple. Just navigate to the Organization network under the Manage and Monitor section, right-click, and select “Reset Network”
    • If you watch vCenter, you will see a new virtual machine get deployed and the old one removed. The process is fairly quick and results in only a minimal outage to running virtual machines using this Edge device

       

Figure 8 – vShield Edge Updating Deployed Edge

 

Figure 9 – Ping Test During vShield Edge Update

  • Examine the new Edge device for the new updated build number that should also match the build number of the vShield Manager.

 

Figure 10 – New vShield Edge Version

  • Updating vApp networks can be achieved the same way; only you need to navigate to the vApp and select “Show Networking Details” to see the network vShield Edge. From there the process is the same, OR the vShield Edge will be updated if the vApp is completely powered off and then powered back on.
  • YOU CAN NOW REDIRECT THE LOAD BALANCER BACK FROM THE MAINTENANCE PAGES AND ALLOW USERS TO ACCESS THE SYSTEM IF YOU CHOOSE TO HAVE THEM ON IT PRIOR TO CHARGEBACK BEING UPGRADED SINCE THE SERVICES ARE STILL DOWN.
  • IF THE CUSTOMER WANTS TO UPGRADE THE ORACLE DATABASE AS WELL AT THIS POINT, DO NOT SEND USERS TO THE PORTAL UNTIL THAT IS COMPLETE.

Upgrading Oracle Database

Since vCloud Director supported on additional versions of Oracle 11g, customers can choose at this point to upgrade the Oracle database server as well. This will also require that the vCloud Cells are not connected to the database, resulting in an outage in most cases and you will need to use the maintenance pages again.

Migrating vCloud Director Database from Oracle to MS SQL Server

This is currently not supported and a SQL Server installation will require a clean database installation and a rerun of the vCloud Director configuration script. There may be unsupported ways of accomplishing this, but the customer would do this at their own risk.

Upgrading vCenter Chargeback

As with all other sections of this document, we will only address vCloud-specific issues when upgrading Chargeback to version 1.6.2. Follow all current documentation for upgrading vCenter Chargeback.

Rolling Back vCenter Chargeback

As with the previous components care should be take to allow a possible back out plan. There are two options for providing a method of rollback it is up to each individual to decide the best method

  • Snapshot the virtual machine – This will require FT is disabled if you had it enabled
  • Backup the Chargeback database

Considerations

  • The installer will require that the previous version be uninstalled. You want to select “do NOT empty the database”.
  • You can uninstall the plug-in, but reinstalling the plug-in will require the Service Account be Administrator in vCenter
  • 1.6.2 is not backward compatible with vCloud Director 1.0.1 so this should be done last in this sequence.
  • There will be database updates as well to support the new vCloud Director data.

Phase I Completion Verification Checklist

DoneRequirement
FSTAB entry changed for new path for NFS Mount
vCloud Director Cells start properly on server reboot
Updated Service Account used for vCloud Director to vShield Manager
Updated Role-based access in vShield Manager
Users can still log into their respective organizations
Load Balancer is properly directing traffic
Verify Host Connectivity
Validate a vApp can be deployed
Validate a new vShield Edge can be deployed
Remove all snapshots of the vCloud Cells and vShield Manager virtual machines

About Chris Colotti

Chris is active on the VMUG and event speaking circuit and is available for many events if you want to reach out and ask. Previously to this he spent close to a decade working for VMware as a Principal Architect. Previous to his nine plus years at VMware, Chris was a System Administrator that evolved his career into a data center architect. Chris spends a lot of time mentoring co-workers and friends on the benefits of personal growth and professional development. Chris is also amongst the first VMware Certified Design Experts (VCDX#37), and author of multiple white papers. In his spare time he helps his wife Julie run her promotional products as the accountant, book keeper, and IT Support. Chris also believes in both a healthy body and healthy mind, and has become heavily involved with fitness as a Diamond Team Beachbody Coach using P90X and other Beachbody Programs. Although Technology is his day job, Chris is passionate about fitness after losing 60 pounds himself in the last few years.

3 comments

  1. upgrade Cells ….. mostly
    eevrything OK ….
    except real world users has GBs of Catalog VMs and
    the upgrade proces copies them to different locations ….
    that time and sopace was not warned in any paper
    …..
    also remember the credntials you need on NFS ..
    other than that … Upgrade OK …..( we are in
    vCD 1.5 in both cells now …)
    vShield Manager Upgrade ….
    WHOA !!!! …….
    TIME …time … and more TIME …. !!!!!
    this is my … SMALL .vcenter … 5 Hosts …
    66 VMs …
    and it is taking WAY MORE than 10 minutes to return
    the connection to the vCenter ….
    right now is more that hour and half ……
    Audit log showing continous “modify” …
    like every 15 or 16 seconds …. so I think is doing something
    but is is painfully slow ….
    really do not want to think what to do tomorrow with my other “BIG” cluster
    …. 12 hosts, 530 VMs ….
    more resources to the vshield Manager ?!?!?

Leave a Reply

Your email address will not be published. Required fields are marked *