Question

Scenario 13‐2: Upgrading a Cluster to Windows Server 2016

As an administrator at the Contoso Corporation, you manage a cluster used as a file server that is running Windows Server 2012 R2. You do not have any other free servers. Describe how to upgrade the cluster to Windows Server 2016.

Answer 1

Cluster Operating System rolling upgrade:

Cluster OS Rolling Upgrade provides the following benefits:

• Failover clusters running Hyper-V virtual machine and Scale-out File Server (SOFS) workloads can be upgraded from Windows Server 2012 R2 (running on all nodes in the cluster) to Windows Server 2016 (running on all cluster nodes of the cluster) without downtime. Other cluster workloads, such as SQL Server, will be unavailable during the time, it takes to failover to Windows Server 2016.

• It doesn't require any additional hardware. Although, you can add additional cluster nodes temporarily to small clusters to improve availability of the cluster during the Cluster OS Rolling Upgrade process.

• The cluster doesn't need to be stopped or restarted.

• A new cluster is not required. The existing cluster is upgraded. In addition, existing cluster objects stored in Active Directory are used.

• The upgrade process is reversible until the customer choses the "point-of-no-return", when all cluster nodes are running Windows Server 2016, and when the Update-ClusterFunctionalLevel PowerShell cmdlet is run.

• The cluster can support patching and maintenance operations while running in the mixed-OS mode.

• It supports automation via PowerShell and WMI.

• The cluster public property ClusterFunctionalLevel property indicates the state of the cluster on Windows Server 2016 cluster nodes. This property can be queried using the PowerShell cmdlet from a Windows Server 2016 cluster node that belongs to a failover cluster

Get-Cluster | Select ClusterFunctionalLevel

A value of 8 indicates that the cluster is running at the Windows Server 2012 R2 functional level. A value of 9 indicates that the cluster is running at the Windows Server 2016 functional level.

Requirements:

Complete the following requirements before you begin the Cluster OS Rolling Upgrade process:

• Start with a Failover Cluster running Windows Server (Semi-Annual Channel), Windows Server 2016, or Windows Server 2012 R2.
• Upgrading a Storage Spaces Direct cluster to Windows Server, version 1709 isn't supported.
• If the cluster workload is Hyper-V VMs, or Scale-Out File Server, you can expect zero-downtime upgrade.
• Verify that the Hyper-V nodes have CPUs that support Second-Level Addressing Table (SLAT) using one of the following methods;
  - Review the Are you SLAT Compatible? WP8 SDK Tip 01 article that describes two methods to check if a CPU supports SLATs
  - Download the Coreinfo v3.31 tool to determine if a CPU supports SLAT.

Cluster transition states during Cluster OS Rolling Upgrade:

This section describes the various transition states of the Windows Server 2012 R2 cluster that is being upgraded to Windows Server 2016 using Cluster OS Rolling Upgrade.

In order to keep the cluster workloads running during the Cluster OS Rolling Upgrade process, moving a cluster workload from a Windows Server 2012 R2 node to Windows Server 2016 node works as if both nodes were running the Windows Server 2012 R2 operating system. When Windows Server 2016 nodes are added to the cluster, they operate in a Windows Server 2012 R2 compatibility mode. A new conceptual cluster mode, called "mixed-OS mode", allows nodes of different versions to exist in the same cluster.

Figure 1: Cluster operating system state transitions

A Windows Server 2012 R2 cluster enters mixed-OS mode when a Windows Server 2016 node is added to the cluster. The process is fully reversible - Windows Server 2016 nodes can be removed from the cluster and Windows Server 2012 R2 nodes can be added to the cluster in this mode. The "point of no return" occurs when the Update-ClusterFunctionalLevel PowerShell cmdlet is run on the cluster. In order for this cmdlet to succeed, all nodes must be Windows Server 2016, and all nodes must be online.

Transition states of a four-node cluster while performing Rolling OS Upgrade:

This section illustrates and describes the four different stages of a cluster with shared storage whose nodes are upgraded from Windows Server 2012 R2 to Windows Server 2016.

"Stage 1" is the initial state - we start with a Windows Server 2012 R2 cluster.

Figure 2: Initial State: Windows Server 2012 R2 Failover Cluster (Stage 1)

In "Stage 2", two nodes have been paused, drained, evicted, reformatted, and installed with Windows Server 2016.

Figure 3: Intermediate State: Mixed-OS mode: Windows Server 2012 R2 and Windows Server 2016 Failover cluster (Stage 2)

At "Stage 3", all of the nodes in the cluster have been upgraded to Windows Server 2016, and the cluster is ready to be upgraded with Update-ClusterFunctionalLevel PowerShell cmdlet.

Figure 4: Intermediate State: All nodes upgraded to Windows Server 2016, ready for Update-ClusterFunctionalLevel (Stage 3)

After the Update-ClusterFunctionalLevelcmdlet is run, the cluster enters "Stage 4", where new Windows Server 2016 cluster features can be used.

Figure 5: Final State: Windows Server 2016 Failover Cluster (Stage 4)

Cluster OS Rolling Upgrade Process:

Cluster OS Rolling upgrade includes the following steps:

1. Prepare the cluster for the operating system upgrade as follows:

   1. Cluster OS Rolling Upgrade requires removing one node at a time from the cluster. Check if you have sufficient capacity on the cluster to maintain HA SLAs when one of the cluster nodes is removed from the cluster for an operating system upgrade. In other words, do you require the capability to failover workloads to another node when one node is removed from the cluster during the process of Cluster OS Rolling Upgrade? Does the cluster have the capacity to run the required workloads when one node is removed from the cluster for Cluster OS Rolling Upgrade?

   2. For Hyper-V workloads, check that all Windows Server 2016 Hyper-V hosts have CPU support Second-Level Address Table (SLAT). Only SLAT-capable machines can use the Hyper-V role in Windows Server 2016.

   3. Check that any workload backups have completed, and consider backing-up the cluster. Stop backup operations while adding nodes to the cluster.

   4. Check that all cluster nodes are online /running/up using the Get-ClusterNode cmdlet (see Figure 7).

f you are running Cluster Aware Updates (CAU), verify if CAU is currently running by using the Cluster-Aware Updating UI, or the Get-CauRun cmdlet (see Figure 8). Stop CAU using the Disable-CauClusterRole cmdlet (see Figure 9) to prevent any nodes from being paused and drained by CAU during the Cluster OS Rolling Upgrade process.Figure 7: Determining node status using Get-ClusterNode cmdletFigure 8: Using the Get-CauRun cmdlet to determine if Cluster Aware Updates is running on the cluster

Figure 9: Disabling the Cluster Aware Updates role using the Disable-CauClusterRole cmdlet

For each node in the cluster, complete the following:

1. Using Cluster Manager UI, select a node and use the Pause | Drain menu option to drain the node (see Figure 10) or use the Suspend-ClusterNode cmdlet (see Figure 11).

Figure 10: Draining roles from a node using Failover Cluster Manager

Figure 11: Draining roles from a node using the Suspend-ClusterNode cmdlet

Using Cluster Manager UI, Evict the paused node from cluster, or use the Remove-ClusterNode cmdlet.

Figure 12: Remove a node from the cluster using Remove-ClusterNode cmdlet

Reformat the system drive and perform a "clean operating system install" of Windows Server 2016 on the node using the Custom: Install Windows only (advanced) installation (See Figure 13) option in setup.exe. Avoid selecting the Upgrade: Install Windows and keep files, settings, and applications option since Cluster OS Rolling Upgrade doesn't encourage in-place upgrade.

Figure 13: Available installation options for Windows Server 2016

1. Add the node to the appropriate Active Directory domain.

2. Add the appropriate users to the Administrators group.

3. Using the Server Manager UI or Install-WindowsFeature PowerShell cmdlet, install any server roles that you need, such as Hyper-V.

Install-WindowsFeature -Name Hyper-V

1. Using the Server Manager UI or Install-WindowsFeature PowerShell cmdlet, install the Failover Clustering feature.

Install-WindowsFeature -Name Failover-Clustering

1. stall any additional features needed by your cluster workloads.

2. Check network and storage connectivity settings using the Failover Cluster Manager UI.

3. If Windows Firewall is used, check that the Firewall settings are correct for the cluster. For example, Cluster Aware Updating (CAU) enabled clusters may require Firewall configuration.

4. For Hyper-V workloads, use the Hyper-V Manger UI to launch the Virtual Switch Manager dialog (see Figure 14).

   Check that the name of the Virtual Switch(s) used are identical for all Hyper-V host nodes in the cluster.

5. Figure 14: Virtual Switch Manager

6. On a Windows Server 2016 node (do not use a Windows Server 2012 R2 node), use the Failover Cluster Manager (see Figure 15) to connect to the cluster.

7. Figure 15: Adding a node to the cluster using Failover Cluster Manager

8. Use either the Failover Cluster Manager UI or the Add-ClusterNode cmdlet (see Figure 16) to add the node to the cluster.

9. Figure 16: Adding a node to the cluster using Add-ClusterNode cmdlet

After the Windows Server 2016 node is successfully added to the cluster, you can (optionally) move some of the cluster workload to the newly added node in order to rebalance the workload across the cluster as follows:

Figure 17: Moving a cluster workload (cluster VM role) using Move-ClusterVirtualMachineRole cmdlet

Use Live Migration from the Failover Cluster Manager for virtual machines or the Move-ClusterVirtualMachineRole cmdlet (see Figure 17) to perform a live migration of the virtual machines.

Move-ClusterVirtualMachineRole -Name VM1 -Node robhind-host3

Use Move from the Failover Cluster Manager or the Move-ClusterGroup cmdlet for other cluster workloads.

When every node has been upgraded to Windows Server 2016 and added back to the cluster, or when any remaining Windows Server 2012 R2 nodes have been evicted, do the following:

Using the Failover Cluster Manager UI or the Get-ClusterGroup cmdlet, check that all cluster roles are running on the cluster as expected. In the following example, Available Storage is not being used, instead CSV is used, hence, Available Storage displays an Offline status (see Figure 18).

Figure 18: Verifying that all cluster groups (cluster roles) are running using the Get-ClusterGroup cmdlet

1. Check that all cluster nodes are online and running using the Get-ClusterNode cmdlet.

2. Run the Update-ClusterFunctionalLevel cmdlet - no errors should be returned (see Figure 19).

3. 

4. Figure 19: Updating the functional level of a cluster using PowerShell

5. After the Update-ClusterFunctionalLevel cmdlet is run, new features are available

Windows Server 2016 - resume normal cluster updates and backups:

If you were previously running CAU, restart it using the CAU UI or use the Enable-CauClusterRole cmdlet (see Figure 20).