CloudPortal Services Manager 11.x High Availability and ... · Manager 11.x for High Availability...

© 2015 Citrix Systems, Inc. All rights reserved

Deploying CloudPortal Services Manager 11.x for High Availability and Disaster Recovery

Last Updated: Mar 23, 2015

CloudPortal Services Manager


Contents Introduction ..................................................................................................................................... 3

Databases ........................................................................................................................................ 3

Pre-requisite ................................................................................................................................ 3

Installation ................................................................................................................................... 3

Adding CPSM Databases to the Availability Group ..................................................................... 4

Login Replication ......................................................................................................................... 5

Recover from the database failure .............................................................................................. 5

Provisioning Engine ......................................................................................................................... 6

Pre-requisites .............................................................................................................................. 6

Installation ................................................................................................................................... 6

Directory Web Service ................................................................................................................... 10

Web Portal and API ....................................................................................................................... 11

HAad Services for App Orchestration ............................................................................................ 12

Reporting ....................................................................................................................................... 13

Other Web Services ....................................................................................................................... 13

Deploy CPSM with App Orchestration HAad Service in multi-datacentre .................................... 14

Multi-Datacentre diagram ..................................................................................................... 14


Introduction CloudPortal Services Manager consists of multiple components:

Core Components:

Databases

Provisioning Engine

Directory Web Service

Web Portal & API

Reporting

Optional Components:

Service-specific web services or tools (e.g. XenDesktop Web Service)

This document provides a basic guidance for deploying these components to support highly

availability and the solution for disaster recovery based on Windows Server 2012, 2012R2, and

SQL Server 2012.

Databases CloudPortal Services Manager uses DNS Alias (CNAME) that points to the SQL server instance

name in connection strings, this simplifies the potential future database move from one SQL

server cluster/group to another.

Pre-requisite SQL Server 2012 AlwaysOn Database Availability Group is configured correctly, enable “SQL

Server and Windows Authentication mode” on each instance and operational following the

Microsoft guidance.

Installation For CPSM to support the SQL server AlwaysOn in a single subnet, first make the CORTEXSQL DNS

alias point to the Listener name as part of the preparation for the initial deployment.

Follow the normal process to start the CloudPortal Services Manager system database installation:

http://support.citrix.com/proddocs/topic/ccps-115/ccps-install-database-create.html. If the

listener configured for the AlwaysOn Availability group is on a port other than 1433, the correct

port number must be specified during the installation step of “Create System Databases”:

http://support.citrix.com/proddocs/topic/ccps-115/ccps-install-database-create.html


Once the installer completes “Create System Databases”, verify that the two CPSM system

databases OLM and OLMReports are restored on the primary replica.

Adding CPSM Databases to the Availability Group Follow the steps below to add the CPSM databases OLM, OLMReports, and OLMReporting

(Reporting database) to the database availability group.

1. Logon to the SQL server that hosts the primary replica and start SQL Server Management

Studio.

2. For each of the 3 databases, change the recovery model from “Simple” to “Full”, and do a

full backup.

3. In Object Explorer, browse and expand the Availability Groups.

4. Right-click the relevant group, and then click Add Database.

5. On the Select Databases page, all databases that are eligible to become the primary

database for the new availability group are listed in the table. The CPSM databases

should be shown as “Meets requirements”. Use the checkboxes to select the 3 CPSM

databases and click Next. Alternatively these databases can be selected and added

individually.

6. On Select Initial Data Synchronization page, accept the default Full option, and Next.

7. If the Validation page displays the results of six checks as successful, click “Next” to

continue. If any test fails, action must be taken to correct the error items and re-run the

validation.

8. On the Summary page, verify the configuration of the replica, and then click Finish.


9. And it’s suggested to set the Availability Mode to Synchronous commit for each replicas

on Availability Group since asynchronous mode may cause the data loss while failure

happen

Login Replication CloudPortal Services Manager SQL logins are not automatically replicated in the availability group

so that need to be manually created on the secondary replicas.

Follow the Method 3 in http://support.microsoft.com/en-us/kb/918992

Connect to secondary replicas by Management Studio, create the account for CortexProp,

OLMUser,OLMReportsUser and OLMReportingUser, the statement will like:

CREATE LOGIN [CortexProp] WITH PASSWORD =

0x02000B50F2F545B3F50C45069BCBFF1598A482E6E4448859D2FEA6C2C43FFAE3CB805E1F7FE7C

C9F6BF7357358B1FDCEFCAC6865327AEAD3452B9D62718516B09ACEA4354278 HASHED, SID =

0x282407CECF437D46A8EEEBC0605F3C7E, DEFAULT_DATABASE = [master], CHECK_POLICY =

ON, CHECK_EXPIRATION = OFF

Where PASSWORD and SID will be the actual one from the output

Recover from the database failure If the hardware/network failure happened on the main datacenter(Node 01 and Node 02), Node

03 in the remote datacenter for example, below is the steps suggested:

1. Administrator connect to the SQL instance of Node 03(remote site) and perform a

forced failover, right click on the availability group Failover…, select the new primary

replica “Node 03”, or it may automatically failover to “Node 03” depend on your

settings, now it will become the primary replica to provide the database services

2. Once the Node 01 and Node 02 comes back online and re-establish the communication

with WSFC cluster, administrator manually resume the database, Resume an Availability

Database (SQL Server)

To resume a secondary database

1. In Object Explorer, connect to the server instance that hosts the availability replica on which you want to resume a database, and expand the server tree.

2. Expand the AlwaysOn High Availability node and the Availability Groups node. 3. Expand the availability group. 4. Expand the Availability Databases node, right-click the database, and click

Resume Data Movement. 5. In the Resume Data Movement dialog box, click OK.

3. Administrator changes the new primary replica to synchronous-commit mode , it will

enable resumed secondary databases to become SYNCHRONIZED,

http://support.microsoft.com/en-us/kb/918992

https://technet.microsoft.com/en-us/library/ff877956(v=sql.110).aspx

https://technet.microsoft.com/en-us/library/ff877956(v=sql.110).aspx


After this step, Node 03’s database will overwrite the one in Node 01 and Node 02,skip

this step if you want to keep Node 01’s database

4. Administrator perform a manual failover to original primary replica(Node 01)

Provisioning Engine CloudPortal Services Manager provisioning engine is dependent on Microsoft Message Queuing,

for high availability requirement, MSMQ needs to be clustered, and so as the CPSM provisioning

engine.

Pre-requisites The provisioning server cluster (Windows Server 2012 Failover Cluster) is created, and in

addition, all servers must be able to see a shared storage device (i.e. a SAN drive) and be able to

take ownership of it. Shared storage is not a requirement for Windows Server 2012 clusters but

is a requirement for some Microsoft services, in this case Microsoft Message Queuing, and port

1801 for Message Queue should not be blocked by the firewall, since Web portal will deliver the

message to provision engine in the cluster by HTTP, now it can only support IPv4, so make sure

IPv4 is your referred protocol, refer to http://support.microsoft.com/en-us/kb/929852 to know

how to disable/enable IPv6 on Web Portal servers

Installation Install and configure CPSM Provisioning role on each of the cluster nodes using the CPSM v11.x

installer: http://support.citrix.com/proddocs/topic/ccps-115/ccps-install-config-roles-gui.html. If

you configure the Provisioning role on the secondary nodes with the same service accounts, make

sure the password matches the ones specified for the same accounts when configuring on the

primary node, and the same Encryption Service can be contacted to retrieve the encrypted key, If

the service cannot be contacted, the Configuration Tool prompts you to import the encrypted key

using a key file. To generate the key file, see Generate and export keyfiles for the Encryption

Service

Follow the steps below to configure the cluster:

1. On the cluster node, open Failover Cluster Manager.

2. Expand Cluster and right click on Roles and select Configure Roles.

3. Click Next and select Message Queuing and click on Next.

http://support.microsoft.com/en-us/kb/929852

http://support.citrix.com/proddocs/topic/ccps-115/ccps-install-config-roles-gui.html

http://support.citrix.com/proddocs/topic/ccps-115/ccps-install-encryption-keygen.html#ccps-install-encryption-keygen

http://support.citrix.com/proddocs/topic/ccps-115/ccps-install-encryption-keygen.html#ccps-install-encryption-keygen


4. Enter the name that the clients will use to access this cluster role. Click Next.

5. Select the shared disk drive name to assign to the cluster role, and Next.


6. Click Finish when “High availability was successfully configured for the role” message and

summary are displayed.

7. Right-click on the newly added role above, and select Add Resource > Generic Service.

8. Select “Citrix Queue Monitor Service” from the list, click Next, follow the wizard and

finish.

9. Right-click the new resource “Citrix Queue Monitor Service”, and select Properties. On

Dependencies tab, add MSMQ-{your new cluster role} as the dependency, and click

Apply.

10. Select General tab, check “Use Network Name for computer name”, and OK. This step

must be performed after step 9, otherwise an error would occur.


11. On each server node, stop “Citrix Queue Monitor Service”, and open Registry.

12. On each server node, Navigate to

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\CortexQueueMonitor,

replace the value off the key “DependOnService” with MSMQ$MSMQ-

{YourClusterRoleName}.

13. On each provisioning server, navigate to the Provisioning Engine folder and edit the

CortexQueueMonitor.exe.config file to point the various queues (request/bulk request

etc) to the cluster name, for example:

<add key=”Message Queue” value=”.\Private$\cortexrequest” /> would become <add

key=”Message Queue” value=”CPSMCluMSMQ\Private$\cortexrequest” />

14. From Failover Cluster Manager, Bring Online “Citrix Queue Monitor Service”.

15. Logon to CloudPortal Services Manager portal as a Service Provider Administrator, go to

Configuration > System Manager > Locations, and expand the relevant location.

16. Change the server name or IP address of the following queue paths to the Message

Queuing cluster role name or IP (not the Windows server cluster name)

and Save.

• Primary Queue Path

• Bulk Queue Path

• Usage Data Queue Path


17. When the preceding changes are complete, restart the CortexDotNet website(s)

(through IIS) and the provisioning engine(s) (through the Failover Cluster Manager) to

reflect the new changes

18. To view the message queue status, requests, and journals for monitoring and

troubleshooting purposes, you can no longer use the local Message Queueing on each of

the nodes, instead, on the host node, from Failover Cluster Manager, select Message

Queuing cluster role, and open Manage Message Queuing…

19. If you want to troubleshooting the Message Queue, you can enable the End2End even log

1. Open Event Viewer on the host node, navigate to Applications and Services

Logs->Microsoft->Windows->MSMQ, right click on End2End, click Enable Log

2. Any sent/received message can be reviewed from End2End panel

Directory Web Service The directory web service is typically located on the same server as the Provisioning Engine and

listens on port 8095, after installed directory web service on each servers, load the global

configuration file and configure it from the Configuration Tool, When the CPSM provisioning


server is clustered, the Directory Web Service is also installed on all cluster nodes (refer to

Provisioning Engine section).

Follow the steps below to configure CPSM to achieve Directory Web Service high availability.

1. Logon to CPSM portal as a service provider administrator.

2. Navigate to Configuration > System Manager > Servers. Click “Refresh Server List”, the

provisioning server cluster name should appear in servers list.

3. If you are configuring for a remote location domain and the DNS of the server names

may not be resolvable in the primary location domain, expand the server cluster, and

enter IP address in Alias field, and Save.

4. Click Server Roles link on the left or got to Configuration > System Manager > Server

Roles, expand the provisioning cluster, tick Directory under Server Connection

Components and Save.

5. Click Server Connections link or go to Configuration > System Manager > Server

Connections, expand the existing entry for the Directory role, select the provisioning

cluster name from the Server dropdown list, and click Save.

6. Test the connection via the icon on the right. It should go green if valid and there no

firewall blocking issues.

Directory Web Service can also be load balanced, in this case the configuration should use the

load balanced VIP address instead of the cluster name/address.

Web Portal and API The web portal (interface) and API are under the same standard .NET framework 4.0 website in

IIS which listens on ports 80 and/or 443. To enable high availability, it is recommended to run

two (or more) nodes in a standard load-balancing setup. Sticky sessions are required.

Installation steps are as below:

1. Install and configure Web server role on all load balanced servers by the same

configuration file: http://support.citrix.com/proddocs/topic/ccps-115/ccps-install-

config-roles-gui.html. If you are adding additional web servers to an existing deployment

with functional services, it is recommended to skip the Service Package Import (deselect

the services and properties) or select Ignore to the properties of all enabled services.




2. The following items of CPSM web on the first/primary web server must be replicated to

all load balanced web servers. It is recommended to replicate all the files in

CortexDotNet and CortexAPI sub sites (except the web.config files specific to the local

sites) from the first/primary web server to all the rest of the web servers.

Images for branding.

Stylesheets for branding.

Any custom downloads.

Any custom DLLs or pages

Web.config configuration changes

IIS Security and authentication changes

And be sure to keep above files synced if you have any configuration changes manully

3. Add the same URL (internal and/or external) host headers to Cortex Management site

on all web servers in IIS Manager, by default it’s CortexWeb

4. Add one DNS records with the IP address of load balanced VIP, and point CORTEXWEB to

it

5. Recycle CortexMgmt application pool via IIS Manager on all web servers.

Steps to configure load balance on Netscaler for Web Portal and API:

1. Login to Netscaler, Traffic Management->Servers, add the Web Portal servers,

Specify the name and IP address

2. Go to Traffic Management->Services to configure the services, add services definition

for https load balancing without SSL offload, in the Protocol, choose “SSL_BRIDGE” and

Choose https as Monitors, for each servers

3. Go to Traffic Management->Virtual Servers, add one virtual server with Protocol

“SSL_BRIDGE” , set Persistence to SOURCEIP to make sure the transaction is in one

same session and activate the services

4. Add one DNS records for the virtual server, and point CortexWeb DNS aliases to this

virtual server

5. If failure happen, restart the browser to open the web console, at this time, another live

Web portal server will provide the service

HAad Services for App Orchestration To provide high availability of HAad services for App Orchestration, at least 2 AO configuration

servers should be installed and configured firstly, then

1. Install and configure HAad service http://support.citrix.com/proddocs/topic/ccps-

115/ccps-install-haad.html on each configuration server

2. Load balancing the AO configuration servers by Netscaler, follow the similar steps

“Steps to configure load balance on Netscaler for Web Portal” mentioned before

3. Navigate to Configuration > System Manager > Servers. Click “Refresh Server List”, if

VIP name does not exist on the list, click Add a Server link, enter the VIP name in the

Server, FQDN in the Alias, the FQDN should match the common name in SSL certificate,

or connection may fail, and click Add Server.

http://support.citrix.com/proddocs/topic/ccps-115/ccps-install-haad.html

http://support.citrix.com/proddocs/topic/ccps-115/ccps-install-haad.html



Roles, expand newly created VIP Server placeholder, tick the “Hosted Apps And

Desktops” role under Server Connection Components and click Save.


Connections, New Connections, select Hosted Apps And Desktops from server role

dropdown list and choose the newly created server , Save



Reporting The high availability of CloudPortal Services Manager Reporting role is dependent on the SQL

Reporting Services HA configurations. The MS SQL Reporting Services achieves HA via a scale-out

deployment so that they share the same report server database:

https://msdn.microsoft.com/en-us/library/bb522745.aspx.

The data source DB (OLMReporting) of CPSM Reports can be added to AlwaysOn availability

group (refer to Adding CPSM Databases to the Availability Group section), in this case when

installing Reporting role via CPSM v11.x installer, the listener name and port should be specified

for the Reporting SQL server.

Other Web Services Similar to the Directory Web Service, other service integration related CPSM web services like

Exchange, Lync, XenDesktop, XenApp, and IIS web services, etc. can be deployed to multiple

servers for high availability:

1. Install and configure the web service on all HA servers using the installer:

http://support.citrix.com/proddocs/topic/ccps-115/ccps-services-deploy.html.

2. Logon to CPSM portal as a service provider administrator to update the web service call

configurations.

3. Navigate to Configuration > System Manager > Servers. Click “Refresh Server List”, if

the cluster or VIP name does not exist on the list, click Add a Server link, enter the VIP

name and click Add Server.

4. If you are configuring for a remote location domain and the DNS of the server names

may not be resolvable in the primary location domain, expand the “server” name you

have just added, enter the IP address in Alias field, and Save.


Roles, expand newly created VIP Server placeholder, tick the appropriate role under

Server Connection Components and click Save.


Connections, expand the existing entry for the web server or connection role to be

https://msdn.microsoft.com/en-us/library/bb522745.aspx

http://support.citrix.com/proddocs/topic/ccps-115/ccps-services-deploy.html


updated, select the newly created VIP Server name from the Server dropdown list, and

click Save.



Deploy CPSM with App Orchestration HAad Service in multi-

datacentre In above sections, we have described how to setup SQL “AlwaysOn” availability group for CPSM,

add Provision Engine and Directory Web service in the cluster, configure load balance for Web

Portal and API and HAad Service for App Orchestration, all these deployment can reside on the

same datacentre to provide high availability, duplicated environment can be deployed on the

remote/secondary datacentre if disaster happen on one datacentre, another datacentre can

function well to provide the services also

To deploy CPSM in the multi-datacentre

1. Database accessibility is guaranteed by the SQL “AlwaysOn” availability group, SQL

replicas are deployed in main and secondary datacentre, and join the same availability

group, all the applications connect to the same database by the same connection string,

to make sure the data is synced timely, set the Availability Mode to Synchronous

commit for each replicas on Availability Group

2. Deploy the Provision Engine servers and Directory Web Service in the same Failover

Cluster on multi-datacenter, and set the server in main datacenter as Owner Node which

will provide the service at normal time, if failure happen, it will switch to secondary

datacentre’s node automatically

3. Deploy Web Portal and API role on multi-datacentre, configure it into one load balance

group, set the LB Method to “Least Response Time” to make sure it’s main datacentre’s

server provide the service at normal time

4. If App Orchestration is deployed in the multi-datacentre and want to enable HAad

service, configure AO configuration servers in the same load balance group, then

configure it on CPSM, this article will not cover how to configure AO in multi-datacentre

Multi-Datacentre diagram

Provision Engine

AO Configuration Servers

Load balancer

Main Datacenter Secondary Datacenter


In above diagram, we deployed 2 servers for each roles in the main Datacenter, it can provide

high availability within the same datacenter, if all the servers are down in one datacenter, load

balancer and Failover cluster can switch it to secondary datacentre also without customer’s

awareness

CloudPortal Services Manager 11.x High Availability and ... · Manager 11.x for High Availability...

Documents

Transcript of CloudPortal Services Manager 11.x High Availability and ... · Manager 11.x for High Availability...