TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307...

24
TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450 Fax: +1 650-846-1005 www.tibco.com TIBCO fuels digital business by enabling better decisions and faster, smarter actions through the TIBCO Connected Intelligence Cloud. From APIs and systems to devices and people, we interconnect everything, capture data in real time wherever it is, and augment the intelligence of your business through analytical insights. Thousands of customers around the globe rely on us to build compelling experiences, energize operations, and propel innovation. Learn how TIBCO makes digital smarter at www.tibco.com. TIBCO ActiveSpaces™ on AKS This document describes how to configure and run the TIBCO ActiveSpaces 4 in an Azure Kubernetes Service (AKS) environment. Version 1.1 July 2019 Initial Document

Transcript of TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307...

Page 1: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

TIBCO Software Inc.

Global Headquarters

3307 Hillview Avenue

Palo Alto, CA 94304

Tel: +1 650-846-1000

Toll Free: 1 800-420-8450

Fax: +1 650-846-1005

www.tibco.com

TIBCO fuels digital business by enabling better decisions and faster, smarter actions through the TIBCO Connected Intelligence Cloud. From APIs and systems to devices and people, we interconnect everything, capture data in real time wherever it is, and augment the intelligence of your business through analytical insights. Thousands of customers around the globe rely on us to build compelling experiences, energize operations, and propel innovation. Learn how TIBCO makes digital smarter at www.tibco.com.

TIBCO ActiveSpaces™ on AKS This document describes how to configure and run the TIBCO ActiveSpaces 4 in an Azure Kubernetes Service (AKS) environment.

Version 1.1 July 2019 Initial Document

Page 2: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 2

Copyright Notice COPYRIGHT© 2019 TIBCO Software Inc. All rights reserved.

Trademarks TIBCO, the TIBCO logo and TIBCO ActiveSpaces are either registered trademarks or trademarks of TIBCO Software Inc. in the United States and/or other countries. All other product and company names and marks mentioned in this document are the property of their respective owners and are mentioned for identification purposes only.

Content Warranty The information in this document is subject to change without notice. THIS DOCUMENT IS PROVIDED "AS IS" AND TIBCO MAKES NO WARRANTY, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING BUT NOT LIMITED TO ALL WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. TIBCO Software Inc. shall not be liable for errors contained herein or for incidental or consequential damages in connection with the furnishing, performance or use of this material.

For more information, please contact:

TIBCO Software Inc. 3303 Hillview Avenue Palo Alto, CA 94304 USA

Page 3: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 3

Table of Contents

1 Overview ................................................................................................................................ 5 1.1 AKS/AS4 Architecture ..................................................................................................................... 5

1.1.1 Development Architecture .......................................................................................................... 5 1.1.2 Production Architecture .............................................................................................................. 5

1.2 Supported Versions ........................................................................................................................ 6 1.3 Prerequisites .................................................................................................................................. 6 1.4 Prepare Local Environment ............................................................................................................. 6 1.5 Prepare Preliminary Azure Account and Kubernetes Configuration ................................................. 7

1.5.1 General (Required) ..................................................................................................................... 7 1.5.2 Configure the Kubernetes Dashboard (Optional) ......................................................................... 7

2 Azure AKS Setup ..................................................................................................................... 8 2.1 Create a New Azure Kubernetes Service (AKS) ................................................................................ 8 2.2 Configuring Kubectl to connect to Azure Kubernetes Service .......................................................... 9

2.2.1 Configure Kubectl to connect to AKS ........................................................................................... 9 2.3 Configure the Azure Container Registry ........................................................................................ 10

2.3.1 Load the TIBCO images into the Docker Registry ....................................................................... 10 2.3.2 Create the Azure Container Registry ......................................................................................... 10 2.3.3 Create a Kubernetes Secret to Access the Individual Azure Container Registry ........................... 11

2.4 Tag and Push the Docker Images to ACR ....................................................................................... 11

3 Configuring ActiveSpaces 4 in AKS ........................................................................................ 13 3.1.1 Configuring AS4 for Kubernetes ................................................................................................ 13 3.1.1 Stopping or Deleting the AS4 processes .................................................................................... 18

4 Accessing and Testing AS4 .................................................................................................... 20 4.1 Accessing AS4 internally ............................................................................................................... 20 4.2 Accessing AS4 Externally ............................................................................................................... 21 4.3 Monitoring ActiveSpaces .............................................................................................................. 22

Page 4: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 4

Table of Figures FIGURE 1 - KUBERNETES CLUSTER CREATION .......................................................................................................................... 9 FIGURE 2 - CONFIGURE KUBECTL ........................................................................................................................................ 9 FIGURE 3 - VERIFY CONNECTING TO THE KUBERNETES CLUSTER ................................................................................................ 10 FIGURE 4 - CREATE ACR REGISTRY .................................................................................................................................... 11 FIGURE 5 - LOGIN INTO THE ACR ...................................................................................................................................... 11 FIGURE 6 - GET ACR CREDENTIALS ................................................................................................................................... 11 FIGURE 7 - CREATE ACR SECRET....................................................................................................................................... 11 FIGURE 8 - LOGIN INTO THE ACR ...................................................................................................................................... 12 FIGURE 9 - TAG AS4 DOCKER IMAGE ................................................................................................................................. 12 FIGURE 10 – AS4-LB-SMALL.YAML EXAMPLE ....................................................................................................................... 15 FIGURE 11 - EXAMPLE OF AS4 STORAGE AND LB PODS RUNNING IN AKS.................................................................................... 15 FIGURE 12 - RUNNING AS4 ENVIRONMENT ......................................................................................................................... 18 FIGURE 13 - TO STOP AND START THE AS4 STATEFULSETS ...................................................................................................... 18 FIGURE 14 - ACCESS TIBDG EXAMPLE ................................................................................................................................. 20 FIGURE 15 - ACCESS OPERATIONS APP EXAMPLE ................................................................................................................... 21 FIGURE 16 - EXTERNAL ACCESS WITH TIBDG EXAMPLE............................................................................................................ 21 FIGURE 17 - ACCESS AS4 EXTERNALLY FROM THE OPERATIONS APPLICATION ............................................................................... 22 FIGURE 18 - FTL MONITOR-START EXAMPLE ........................................................................................................................ 23 FIGURE 19 - IMPORTING THE AS4 DASHBOARDS ................................................................................................................... 23 FIGURE 20 - ACTIVESPACES GIRD ACTIVITY DASHBOARD ........................................................................................................ 24

Page 5: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 5

1 Overview

This document will outline how to configure the TIBCO ActiveSpaces® v4 in a Kubernetes cluster on Azure. The Kubernetes cluster will be built using Azure Kubernetes Service (AKS). Running TIBCO AS4 on Azure involves:

• Configuring the Azure Kubernetes Service (AKS) for TIBCO ActiveSpaces. • Configuring the Azure Container Registry (ACR) for the Docker® image registry, and

hosting the AS4 and FTL5.4 Docker images in ACR. • Configuring and creating Kubernetes containers based on the Docker images for the

individual components

1.1 AKS/AS4 Architecture

Using this document, two different architectures can be produced. A smaller configuration suitable for development/testing, or a larger fault-tolerant configuration suitable for production. Essentially, they are the same, with the larger environment consisting of a F/T Realm Server, and additional copy sets, nodes, and proxies.

1.1.1 Development Architecture The development architecture created will contain:

• One (1) VPC • AKS cluster will consist of eight (8) nodes with a minimum of 16 GB of RAM and 4 CPUs. • Azure managed premium disks for all configuration and data. • ACR Registries for all containers • ELB (Kubernetes) for external access to the Realm Services and the AS4 Proxies. • One (1) AS4 Copyset • Two (2) AS4 Nodes (1 – replica set) • Three (3) AS4 Statekeepers • Two (2) AS4 Proxies • One (1) FTL Realm Server

1.1.2 Production Architecture The created architecture created will contain:

• One (1) VPC • AKS cluster will consist of ten (10) nodes with a minimum of 16 GB of RAM and 4 CPUs.

Larger cluster nodes can be used. • Azure managed premium disks for all nodes in the cluster • Azure managed premium disks for the AS4 persisted disks • ACR Registries for all containers • ELB (Kubernetes) for external access to the Realm Services and the AS4 Proxies. • Two (2) AS4 Copysets • Four (4) AS4 Nodes (2 – replica sets)

Page 6: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 6

• Three (3) AS4 Statekeepers • Four (4) AS4 Proxies • Two (2) FTL Realm Servers – (FTL 5.4 Primary and Secondary configuration)

1.2 Supported Versions

The steps described in this document are supported for the following versions of the products and components involved:

• TIBCO ActiveSpaces 4.1 • TIBCO FTL 5.4.1 • Docker Community/Enterprise Edition should be most recent version, (at least 18.09.2), to

address recent security vulnerabilities • Kubernetes 1.12 or newer

1.3 Prerequisites

The reader of this document must be familiar with:

• Docker concepts • Azure console, the Azure CLI (az) • Kubernetes installation and administration • Kubernetes CLI, kubectl • TIBCO AS4 installation and configuration • All necessary downloads discussed in the next section • The appropriate TIBCO license(s), if required.

1.4 Prepare Local Environment

General:

The following infrastructure should already be in place:

• A local Linux or macOS machine equipped for building Docker images • The following software must already be downloaded to the Linux or macOS machine

equipped for building Docker images.

Note: All software must be for Linux!

• TIBCO ActiveSpaces v4.1 Docker images which are part of the TIBCO AS4 installation package. The Enterprise Edition or the Community Edition can be used. Download the EE from edelivery.tibco.com , and the CE at https://www.tibco.com/products/messaging-event-processing

• The tibas4_aks_files.zip. The zip file contains all of the necessary Kubernetes build files. Download from https://community.tibco.com/wiki/tibcor-messaging-article-links-quick-access

• Create a directory, place tibas4_aks_files.zip in the directory. • Unzip tibas4_aks_files.zip.

Page 7: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 7

• Copy all of the as-*.xz and ftl-*.xz Docker images from the TIBCO AS4 installation package to tibas4_aks_files/docker directory.

1.5 Prepare Preliminary Azure Account and Kubernetes Configuration

Use the following to prepare the preliminary environment to install the TIBCO AS4 on AKS.

1.5.1 General (Required) • An active Azure account is required. If necessary, create an account at

http://portal.azure.com and follow the on-screen instructions.

• Install the Azure CLI on the workstation used.

• Install Docker on the workstation to load the TIBCO AS4 images.

• Install the kubectl command-line tool to manage and deploy applications to Kubernetes in AZURE from a workstation.

1.5.2 Configure the Kubernetes Dashboard (Optional) The Kubernetes Dashboard is a general purpose, web-based UI for Kubernetes clusters. It allows users to manage applications running in the cluster and troubleshoot them. Note: This assumes the local workstation used to configure AKS is used. If running on an Azure VM or other workstation, kubectl must be installed. Access will also differ, as localhost cannot be used. Use the following to setup un the dashboard:

• Issue the following command to deploy the Kubernetes Dashboard: kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml

• Run kubectl proxy & - to run the proxy in the background, or open a second terminal shell, and run kubectl proxy

• Open web browser and access: http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/.

• Use the Authentication link to setup a Service Account Token to access the Dashboard.

Page 8: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 8

2 Azure AKS Setup

2.1 Create a New Azure Kubernetes Service (AKS)

A new Kubernetes cluster must be created in AKS. Use the following to build a new Kubernetes Service in Azure. This can be created via the Azure Portal of the Azure CLI. This document will outline building the cluster via the Azure portal.

• Sign into the Azure portal at https://portal.azure.com/ • In the top left-hand corner of the Azure portal, select Create a resource > Kubernetes

Service. • Select a Subscription and Resource group. A new or existing Resource group can be used.

The same Resource group must be used throughout. • Provide a new Kubernetes Cluster Name, Region (use the same as for the ACR),

Kubernetes version (must be at least 1.12.7), and a DNS name prefix, such as as4. • For Scale, select the node size. Recommend a B4MS (4 CPU / 16 GB RAM) • Select a node count of (8 or 10), depending on if you are building a development or

production environment • Leave virtual nodes disabled or enabled based on requirements. VM scale sets should be

enabled (when available). Enabling virtual nodes is not required. • Click on Next: Authentication • Select to create a new service principal • Click on Yes to Enable RBAC • Click on Next: Networking • Choose either Yes or No for application routing, depending on requirements • Choose either Basic or Advanced for Network configuration. Recommend using Basic. • Monitoring should be set based on requirements • Wait for the Running the Validation to complete, with validation passed. Fix any issues

before continuing! • Click on Create. It will take several minutes to complete.

Page 9: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 9

Figure 1 - Kubernetes Cluster creation

2.2 Configuring Kubectl to connect to Azure Kubernetes Service

With AKS, the Kubernetes command line tool, kubectl, is used to configure the Kubernetes cluster for AS4 on AKS.

2.2.1 Configure Kubectl to connect to AKS After the Kubernetes cluster has been built, kubectl must be configured to connect to the cluster on AKS. Use the following example to set the credentials for kubectl.

Figure 2 - Configure Kubectl

Use kubectl get nodes as shown in the following example to verify connecting to the cluster.

Page 10: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 10

Figure 3 - Verify connecting to the Kubernetes Cluster

2.3 Configure the Azure Container Registry

New ACR registry(s) must be created to host the TIBCO AS4/FTL 5.4 Docker images. Use this section to create the necessary ACR registry(s). Seven (7) Docker images need to be tagged and pushed to ACR. All images can be in the same registry, or separated in to individual registries. This document will create one registry for all Docker images. This is mainly due to the Kubernetes acr-secret that is required.

2.3.1 Load the TIBCO images into the Docker Registry Load the the TIBCO AS4/FTL Docker images into the local Docker registry.

• Change directory to the directory where the AS4/FTL Docker images are located. • Load the the Docker images in the local registry:

docker load -i as-tibdg-4.1.0.dockerimage.xz docker load -i as-tibdgnode-4.1.0.dockerimage.xz docker load -i as-tibdgkeeper-4.1.0.dockerimage.xz docker load -i as-tibdgproxy-4.1.0.dockerimage.xz docker load -i as-tibdgadmind-4.1.0.dockerimage.xz docker load -i as-operations-4.1.0.dockerimage.xz docker load -i ftl-tibrealmserver-5.4.1.dockerimage.xz

2.3.2 Create the Azure Container Registry • Create the new ACR registry, such as as410. The registry can be created via the Azure CLI

or via the console. Please note the loginServer of your ACR registry. Create the registry using the example below. Then, create additional registries, if required.

Page 11: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 11

Figure 4 - Create ACR Registry

• Login into the newly created Azure ACR from the Azure CLI.

Figure 5 - Login into the ACR

2.3.3 Create a Kubernetes Secret to Access the Individual Azure Container Registry Kubernetes needs credentials to access the Azure Container Registry used for the Docker images. The credentials are stored in a Kubernetes secret. The secret key will be referenced with the appropriate TIBCO AS4 component. Use the Azure CLI to get the credentials.

• Use the following to get the ACR credentials. Substitute the appropriate resource group and registry name.

Figure 6 - Get ACR Credentials

• Create the acr-secret with kubectl with the following command. This must be done for each of the registries created (note recommended). Substitute the appropriate name of the registry for <registry-name>. Use the admin password (either 1 or 2) from the previous step. The AS4 yaml files will reference as410-acr-secret as the ACR secret. If is recommended to use as410-acr-secret for the secret, or the yaml files will need to be modified.

Figure 7 - Create ACR Secret

2.4 Tag and Push the Docker Images to ACR

Once the ACR is ready, the Docker images can be tagged and pushed to ACR. Tag the image and push the Docker images to the ACR registry using the URL of the appropriate registry

• Login into the newly created Azure ACR from the Azure CLI.

Page 12: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 12

Figure 8 - Login into the ACR

• Tag the image and push the Docker image to the ACR repository using the loginServer name noted in section 2.3.2. The loginServer can also be retrieved from the Azure console under your Resource Group. Note: Name of Docker image may differ depending on setup.

Figure 9 - Tag AS4 Docker image

• Push the AS4 Docker image to ACR. Replace the name of the loginServer

• Tag and push the remaining six Docker images to the same ACR, using different extensions for each to identify the image, such as tibdg, oper, proxy, keeper, node, admind, and ftl54.

Page 13: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 13

3 Configuring ActiveSpaces 4 in AKS

After the FTL and AS4 Docker images are pushed to ACR, Kubernetes can be configured to run the AS4/FTL containers.

3.1.1 Configuring AS4 for Kubernetes There are different templates used depending on if the development or production environment is being configured. In the tibas4_aks_files/kubernetes directory, there will be five templates; two for the development environment (small), and two for the production environment (large), and a fifth template, as4-storage.yaml, which is used by both to create the required persisted storage. The as4-storage.yaml file will create the persisted storage as managed premium-LRS. This can be modified for geo-redundant storage, but can only be standard. See the Azure documentation for details. The as4-aks-small.yaml and the as4-aks-large.yaml are used to create the bulk of the environment. The as4-lb-small.yaml and the as4-lb-large.yaml will create the Kubernetes load balancers for the environment. The only difference with the load balancer files is the number of Realm Server and proxy load balancers created. The main configuration file and the LB configuration file will need minor changes. The storage configuration file requires no modifications.

3.1.1.1 Load Balancer Configuration Following, is the tibas4_aks_files/kubernetes/as4-lb-small.yaml used to configure the load balancers. The are three load balancers created for the development environment, while there six created in the production environment. The change required is exactly the same for both. The trusted IP range will determine what IP addresses can connect to the load balancer, and should be configured. NOTE: The load balancer file must be configured and applied first. No other changes should be made or than the trusted IP range! If port number changes are made, the port changes must also be made in the as4-aks yaml file. It will take a few minutes for all of the load balancers to be created.

apiVersion: v1 kind: Service metadata: labels: name: realmserver-lb name: realmserver-lb spec: externalTrafficPolicy: Cluster ports: - name: realm nodePort: 30080 port: 30080 targetPort: 30080 protocol: TCP

Page 14: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 14

- name: admind nodePort: 30081 port: 30081 targetPort: 30081 protocol: TCP - name: ftl nodePort: 30083 port: 30083 targetPort: 30083 protocol: TCP selector: com.tibco.datagrid.service: realmserver sessionAffinity: None type: LoadBalancer loadBalancerSourceRanges: - <your trusted IP range in the form of 0.0.0.0/0> (1) status: loadBalancer: {} --- apiVersion: v1 kind: Service metadata: labels: app: tibdg name: proxy-0-lb name: proxy-0-lb spec: externalTrafficPolicy: Cluster ports: - name: proxy-0 nodePort: 30086 port: 30086 targetPort: 8555 protocol: TCP selector: com.tibco.datagrid.service: tibdgproxy sessionAffinity: None type: LoadBalancer loadBalancerSourceRanges: - <your trusted IP range in the form of 0.0.0.0/0> (1) status: loadBalancer: {} --- apiVersion: v1 kind: Service metadata: labels: app: tibdg name: proxy-1-lb name: proxy-1-lb spec: externalTrafficPolicy: Cluster ports:

Page 15: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 15

- name: proxy-1 nodePort: 30087 port: 30087 targetPort: 8555 protocol: TCP selector: com.tibco.datagrid.service: tibdgproxy sessionAffinity: None type: LoadBalancer loadBalancerSourceRanges: - <your trusted IP range in the form of 0.0.0.0/0> (1) status: loadBalancer: {}

Figure 10 – as4-lb-small.yaml Example

(1): The trusted IP address range to connect to the load balancer.

3.1.1.2 Applying the LB and Storage configurations in Kubernetes Once the AS4 LB yaml files has been updated, this file, along with the storage yaml files can be applied using kubectl to AKS. Use kubectl apply –f as4-storage.yaml,as4-lb-small.yaml to apply both files. Use kubectl get storageclass,svc to verify the storage classes and load balancers are available. Do not continue until the load balancers have been assigned an External IP address as shown in the following example. These IP addresses are required for the main AS4 yaml file.

Figure 11 - Example of AS4 storage and LB pods running in AKS

3.1.1.3 AS4 configuration Once there are load balancer IP addresses for the proxies, the main configuration file as4-aks-small.yaml can be configured. On the again, changes to the as4-aks-large.yaml files are the same, just additional locations for modification. The only changes required is to the proxy_client_listen_external_host for proxy-0 and proxy-1 and the ACR image for each container.

Page 16: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 16

--- # This is used to populate the grid configuration. apiVersion: v1 kind: ConfigMap metadata: name: tibdg-conf labels: app: tibdg data: conf.tibdg: | grid create copyset_size=2 proxy_client_listen_port=8555 copyset create cs-01 node create --copyset cs-01 --dir /data/cs-01-node-0 cs-01-node-0 node create --copyset cs-01 --dir /data/cs-01-node-1 cs-01-node-1 keeper create --dir /data/keeper-0 keeper-0 keeper create --dir /data/keeper-1 keeper-1 keeper create --dir /data/keeper-2 keeper-2 proxy create proxy_mirroring_listen_port=8556 proxy_client_listen_external_host=<proxy-0 external IP address> proxy_client_listen_external_port=30086 proxy-0 (1) proxy create proxy_mirroring_listen_port=8557 proxy_client_listen_external_host=<proxy-1 external IP address> proxy_client_listen_external_port=30087 proxy-1 (1) table create t1 key long column create t1 value string ... --- apiVersion: apps/v1 kind: StatefulSet metadata: name: realmserver labels: com.tibco.datagrid.service: realmserver app: tibdg spec: serviceName: realmserver replicas: 1 selector: matchLabels: com.tibco.datagrid.service: realmserver template: metadata: name: realmserver labels: com.tibco.datagrid.service: realmserver app: tibdg spec: containers: - name: realmserver

Page 17: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 17

image: <your ACR registry for the FTL 5.4 realm server container> (2) imagePullPolicy: Always resources: requests: cpu: 1 memory: 4Gi limits: cpu: 2 memory: 8Gi args: - '--ftl' - '*:30083' - '--gui' - '*:30085' - '--http' - '*:30080' volumeMounts: - mountPath: /data name: realmserver-data - name: tibdgadmind image: <your ACR registry for the AS4 admind container> (2) imagePullPolicy: Always resources: requests: cpu: 1 memory: 4Gi limits: cpu: 2 memory: 8Gi args: [ '-r', 'http://realmserver:30080', '-l', ':30081' ] imagePullSecrets: - name: as410-acr-secret (3) volumeClaimTemplates: - metadata: name: realmserver-data spec: accessModes: - ReadWriteOnce storageClassName: as4-data resources: requests: storage: 5Gi

(1): The proxy_client_listen_external_host. For the the development (small) configuration, this is proxy-0 and proxy-1. For the large configuration, this will also include proxy-2 and proxy-3. Insert the value of the External IP for the matching load balancer. An example would be 40.124.10.20.

Page 18: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 18

(2): The name and location of the Azure Container Registry (ACR) where the FTL/AS4 Docker containers are located. Ensure the proper permissions are set. The image maybe something different than latest, depending on how it was tagged in Docker. (3): The ACR secret. If not using as410-acr-secret, change to correct name of the secret. Note: If port numbers or other changes are made, the port number changes must also be made in the as4-lb yaml file. If is not recommended to make other changes unless, you are very familiar with Kubernetes. After making the necessary modifications, use kubectl apply –f as4-aks-small.yaml to apply the configuration to Kubernetes. Note: This will take a few minutes, (~4 minutes) and there will be some pod restarts. This is normal. Use kubectl get pods to get the status pf the pods. Wait until all pods are running, and the tibdgconfig pod has completed before continuing.

Figure 12 - Running AS4 environment

3.1.1 Stopping or Deleting the AS4 processes To stop all of the processes without deleting them, use the kubectl scale operation to set its number of replicas to 0. For example: > kubectl scale --replicas=0 statefulset proxy,keeper,cs-01-node,realmserver

To start the process again, set its number of replicas back to their original values. Node is two, keeper is three, proxy is two, and realmserver is one. For example: > kubectl scale --replicas=3 statefulset keeper

Figure 13 - To Stop and Start the AS4 Statefulsets

To delete the all of the statefulsets and services entirely, use the kubectl delete operation: kubectl delete –f as4-aks-small.yaml,as4-lb-small.yaml,as4-storage.yaml

Page 19: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 19

All of the corresponding pods, statefulsets, storage classes, and LB services will be deleted. The PVC and PV will not be deleted, nor will the corresponding data. To delete the data, PV, and PVC, use the following: Kubectl delete pvc,pv --all

Page 20: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 20

4 Accessing and Testing AS4

This section will outline testing ActiveSpaces 4 running in AKS. Access can be tested internally and externally.

4.1 Accessing AS4 internally

Testing access internally can be done by trying to access tibdg and running the operations sample application. The External IP Address the realmserver-lb is all that is needed. Use: docker run –rm –it as-tibdg:4.1.0 –r http://<IP of RS LB>:30080 status. Substitute the location of the as-tibdg:4.1.0 for your Docker registry, and your Realm Server LB external address.

Figure 14 - Access Tibdg example

To test with the operations apps. Use the following: docker run –rm -it as-operations:4.1.0 –r http://<IP of RS LB>:30080

Page 21: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 21

Figure 15 - Access Operations app example

4.2 Accessing AS4 Externally

In some instances, accessing ActiveSpaces will be from an external site. To do this test, ActiveSpaces 4 must be installed on a workstation. You will still need the External IP Address for the Realm Server LB. To test with tibdg:

• Change directory to /opt/tibco/as/4.1/bin • Run ./tibdg –r http://<external IP of the RS LB>:30080 status. You should see something

similar to the following example.

Figure 16 - External Access with Tibdg example

Testing with the operations application is similar: • Change directory to /opt/tibco/as/4.1/samples/bin

Page 22: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 22

• Run ./operations –r http://<external IP for the RS LB>. You should get the operations menu. Feel free to add/get/delete data. This will verify that AS4 is active and available from an external source.

Figure 17 - Access AS4 externally from the Operations application

4.3 Monitoring ActiveSpaces

AS4 can be monitored utilizing the FTL5.4/AS4.1 monitoring utilities from an external location. You will need the External IP Address for the Realm Server LB.

Note: The Enterprise edition of FTL 5.4 and AS 4.1 are required for the monitoring components.

• On the workstation that was used to build the AKS environment, install both FTL5.4 and ActiveSpaces 4.1 following their respective installation guides.

• Once both products are installed, change directory to /opt/tibco/ftl/5.4/monitoring • Run ./monitor-start.py operations –r http://<external IP for the RS LB>.

Note: On a Mac (Darwin 18.6.0) it may be necessary to modify monitor-start.py, and remove releases from line3. You should get output similar to the following example:

Page 23: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 23

Figure 18 - FTL monitor-start example

• Change directory to /opt/tibco/as/4.1/monitor/scripts, and run ./import-activespaces-dashboards. This will import the AS4 dashboards into the FTL monitor. This only has to be done once. The output should be similar to the following example:

Figure 19 - Importing the AS4 dashboards

• Using a web browser, go to http://<machine running the monitoring software>:3000 • Login in using admin/admin. The user password can be changed, as well as securing the

monitor. See the FTL 5.4 and ActiveSpaces 4.1 documentation for details. • Click on the Home to get the Dashboards, and then click on the ActiveSpaces Grid Activity

dashboard. You should get something similar to the following example:

Page 24: TIBCO ActiveSpaces4 on K8 with AKS · 2019-07-20 · TIBCO Software Inc. Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 Toll Free: 1 800-420-8450

©2019 TIBCO Software Inc. All Rights Reserved. 24

Figure 20 - ActiveSpaces Gird Activity Dashboard

• Feel free to use the operations sample application to enter data into the grid, and see the rates change.

• Use monitor-stop.py to stop the monitoring software.