Table of ContentsLab Overview - HOL-2101-05-CMP - Troubleshooting and Remediation with vRealizeOperations and Log Insight ............................................................................................... 2
Lab Guidance .......................................................................................................... 3Module 1 - You just received a call about a virtual machine’s performance, now what?(45 minutes) ..................................................................................................................... 9
Introduction........................................................................................................... 10Log in to the vRealize Operations ......................................................................... 12Troubleshooting a VM. Why is the web page down?.............................................. 16Conclusion............................................................................................................. 39
Module 2 - Utilizing vRealize Log Insight to find issues in our environment (45minutes).......................................................................................................................... 41
Introduction........................................................................................................... 42Log in to the vRealize Operations ......................................................................... 43vRealize Operations and vRealize Log Insight Better Together .............................47Conclusion............................................................................................................. 68
Module 3 - Your application team is asking what vRealize Operations can do to helpmonitor their application (45 minutes) ........................................................................... 69
Introduction........................................................................................................... 70Hands-on Labs Interactive Simulation: Your application team is asking whatvRealize Operations can do to help monitor their application...............................71Conclusion............................................................................................................. 72
Module 4 - Determine the health of our vRealize Operations environment (30 minutes)73Introduction........................................................................................................... 74Log in to the vRealize Operations HVM Instance................................................... 75Using Dashboards to Monitor Health..................................................................... 77Using Alerts for vRealize Operations Health.......................................................... 83Conclusion............................................................................................................. 88
Module 5 - Using vRealize Operations for vSAN dashboards to evaluate and managestorage (30 minutes) ...................................................................................................... 90
Introduction........................................................................................................... 91Log in to the vRealize Operations HVM Instance................................................... 92Troubleshooting vSAN ........................................................................................... 94vSAN Capacity Management............................................................................... 101Hands-on Labs Interactive Simulation: Using vRealize Operations for vSANdashboards to evaluate and manage storage..................................................... 109Customize Cost Drivers....................................................................................... 110Conclusion........................................................................................................... 114
HOL-2101-05-CMP
Page 1HOL-2101-05-CMP
Lab Overview -HOL-2101-05-CMP -Troubleshooting and
Remediation withvRealize Operations and
Log Insight
HOL-2101-05-CMP
Page 2HOL-2101-05-CMP
Lab GuidanceNote: It will take more than 90 minutes to complete this lab. You shouldexpect to only finish 2-3 of the modules during your time. The modules areindependent of each other so you can start at the beginning of any moduleand proceed from there. You can use the Table of Contents to access anymodule of your choosing.
The Table of Contents can be accessed in the upper right-hand corner of theLab Manual.
Enable self-driving operations by using vRealize Operations and vRealize Log Insight totroubleshoot infrastructure and application issues in your vSphere environment andthen automatically remediate those issues using vRealize Orchestrator workflows. Youwill also learn how to use several dashboards to monitor, troubleshoot, manage, andoptimize your hyperconverged infrastructure powered by vSAN, your applications, andeven your vRealize Operations environment.
Lab Module List:
• Module 1 - You just received a call about a virtual machine’s performance, nowwhat? (45 minutes) (Basic) Using vRealize Operations you can diagnose the issueand then setup the proper alerts to be notified as performance begins to sufferand correct the issue automatically.
• Module 2 - Utilizing vRealize Log Insight to find issues in our environment (45minutes) (Basic) Use vRealize Operations and vRealize Log Insight together toidentify the root cause of an issue and then configure an automated remediationof the issue using a custom vRealize Orchestrator workflow.
• Module 3 - Your application team is asking what vRealize Operations can do tohelp monitor their application (45 minutes) (Basic) In this chapter, we will explorea scenario where the Infrastructure Administrator at Rainpole Company receivesa report from an Application Owner about a malfunctioning website.Infrastructure teams commonly own vRealize Operations and are not aware ofapplication level visibility in the tool. In this use case, we will walk them throughhow they can monitor an application.
• Module 4 - Determine the health of our vRealize Operations environment (30minutes) (Basic) We typically utilize vRealize Operations to support other systemsbut what happens when it has an issue. In this section we will walk through thebasics of where to look for help.
• Module 5 - Using vRealize Operations for vSAN dashboards to evaluate andmanage storage (30 minutes) (Basic) In this module, we will learn how to makevSAN operational in a production environment by using dashboards to evaluate,manage, and optimize the performance of vSAN objects and vSAN-enabledobjects in vRealize Operations.
HOL-2101-05-CMP
Page 3HOL-2101-05-CMP
Lab Captains:
• Module 1 - Tony Welsh, Staff Cloud Management Specialist, USA• Module 2 - Tony Welsh, Staff Systems Engineer Specialist, USA• Module 3 - Sofia Blasini, Associate Solution Engineer, USA• Module 4 - Sofia Blasini, Associate Solution Engineer, USA• Module 5 - Sofia Blasini, Associate Solution Engineer, USA
This lab manual can be downloaded from the Hands-on Labs Document site found here:
http://docs.hol.vmware.com
This lab may be available in other languages. To set your language preference and havea localized manual deployed with your lab, you may utilize this document to help guideyou through the process:
http://docs.hol.vmware.com/announcements/nee-default-language.pdf
Location of the Main Console
1. The area in the RED box contains the Main Console. The Lab Manual is on the tabto the Right of the Main Console.
2. A particular lab may have additional consoles found on separate tabs in the upperleft. You will be directed to open another specific console if needed.
HOL-2101-05-CMP
Page 4HOL-2101-05-CMP
3. Your lab starts with 90 minutes on the timer. The lab can not be saved. All yourwork must be done during the lab session. But you can click the EXTEND toincrease your time. If you are at a VMware event, you can extend your lab timetwice, for up to 30 minutes. Each click gives you an additional 15 minutes.Outside of VMware events, you can extend your lab time up to 9 hours and 30
minutes. Each click gives you an additional hour.
Alternate Methods of Keyboard Data Entry
During this module, you will input text into the Main Console. Besides directly typing itin, there are two very helpful methods of entering data which make it easier to entercomplex data.
Click and Drag Lab Manual Content Into Console ActiveWindow
You can also click and drag text and Command Line Interface (CLI) commands directlyfrom the Lab Manual into the active window in the Main Console.
Accessing the Online International Keyboard
You can also use the Online International Keyboard found in the Main Console.
1. Click on the Keyboard Icon found on the Windows Quick Launch Task Bar.
<div class="player-unavailable"><h1 class="message">An error occurred.</h1><div class="submessage"><ahref="http://www.youtube.com/watch?v=xS07n6GzGuo" target="_blank">Try watching this video on www.youtube.com</a>, or enableJavaScript if it is disabled in your browser.</div></div>
HOL-2101-05-CMP
Page 5HOL-2101-05-CMP
Click once in active console window
In this example, you will use the Online Keyboard to enter the "@" sign used in emailaddresses. The "@" sign is Shift-2 on US keyboard layouts.
1. Click once in the active console window.2. Click on the Shift key.
Click on the @ key
1. Click on the "@ key".
Notice the @ sign entered in the active console window.
HOL-2101-05-CMP
Page 6HOL-2101-05-CMP
Activation Prompt or Watermark
When you first start your lab, you may notice a watermark on the desktop indicatingthat Windows is not activated.
One of the major benefits of virtualization is that virtual machines can be moved andrun on any platform. The Hands-on Labs utilizes this benefit and we are able to run thelabs out of multiple datacenters. However, these datacenters may not have identicalprocessors, which triggers a Microsoft activation check through the Internet.
Rest assured, VMware and the Hands-on Labs are in full compliance with Microsoftlicensing requirements. The lab that you are using is a self-contained pod and does nothave full access to the Internet, which is required for Windows to verify the activation.Without full access to the Internet, this automated process fails and you see this
watermark.
This cosmetic issue has no effect on your lab.
Look at the lower right portion of the screen
HOL-2101-05-CMP
Page 7HOL-2101-05-CMP
Please check to see that your lab is finished all the startup routines and is ready for youto start. If you see anything other than "Ready", please wait a few minutes. If after 5minutes your lab has not changed to "Ready", please ask for assistance.
HOL-2101-05-CMP
Page 8HOL-2101-05-CMP
Module 1 - You justreceived a call about a
virtual machine’sperformance, now what?
(45 minutes)
HOL-2101-05-CMP
Page 9HOL-2101-05-CMP
IntroductionAt Rainpole corporation release 4 of the Moad Electronic Store Application is notfunctioning and you need to get this resolved quickly. In this module you will usevRealize Operations to investigate why the application is not functioning and fix theissue.
This Module contains the following lessons:
• How to diagnose problems with the Virtual Machine troubleshooting dashboard• Using Alerts• Use Metric charts• Getting Top Processes• Execute actions
Module Setup
There is one step to initiate before starting the module.
1. Click the Lab Files folder on the desktop to open it.2. Double-Click 2101-05-M1-STARTUP. A program is started and is running
minimized.
Start Module Minimized
1. You should see the 2101-05-M1-STARTUP program running in the task bar asshown here.
HOL-2101-05-CMP
Page 10HOL-2101-05-CMP
Close Lab Files Folder
1. Click the X to close the folder. Now you are ready to start the module.
HOL-2101-05-CMP
Page 11HOL-2101-05-CMP
Log in to the vRealize OperationsThis lab environment is running two different instances of vRealize Operations. We havethe different vRealize Operations instances for different use cases. The lab instancesare:
• Live Instance: Connected to the small running vSphere environment in the lab.There isn't a large inventory of objects in this instance but it allows us to interactwith a live system including vCenter.
• Historical View Mode (HVM) Instance: Running a time loop of data that wascaptured in the past. This instance has a much larger inventory of objects but,since it is not currently connected to a vCenter Server, we cannot perform anyactions here.
In this lesson we will be using the live Instance of vRealize Operations.
If you are not currently logged into any instance of vRealize Operations, continue to thenext page, but if you are already logged into the live (not historical) instance ofvRealize Operations, click to skip ahead.
Open the Chrome Browser from Windows Quick LaunchTask Bar
If your browser isn't already open, launch Google Chrome
1. Click the Chrome icon on the Windows Quick Launch Task Bar
The browser Bookmarks Bar has links to the different instances of vRealize Operationsthat are running in the lab.
1. The Live instance of vRealize Operations has already been started for you.
HOL-2101-05-CMP
Page 12HOL-2101-05-CMP
Log in to vRealize Operations
vRealize Operations is integrated with VMware Identity Manager which we will use foruser authentication in this lab.
VMware Identity Manager should be pre-selected as the identity source. However, if it isnot you will choose it.
1. Click the drop-down arrow if VMware Identity Manager is not selected.2. Click REDIRECT to be taken to the authentication page.
HOL-2101-05-CMP
Page 13HOL-2101-05-CMP
VMware Identity Manager Login
For the Live instance of vRealize Operations instance, the default username andpassword should already be entered. However, if needed type them in.
username: holadmin
password: VMware1!
1. Click Sign in
HOL-2101-05-CMP
Page 14HOL-2101-05-CMP
vRealize Operations Home Screen
You should be at the vRealize Operations Home screen and ready to start the module.
HOL-2101-05-CMP
Page 15HOL-2101-05-CMP
Troubleshooting a VM. Why is the webpage down?As a member of the operations team at Rainpole, you are responsible for maintainingapplication uptime within the environment. Just as you get to your desk in the morningyou get a frantic call from the VP of Customer Service saying the shopping cartapplication is not functioning and it needs to be fixed ASAP.
The application is the Moad Electronic store application and the initial web page shouldlook like the image on the next page.
MOAD Electronic Super
The MOAD application consists of 2 virtual machines, Moad-Web and Moad-SQL. Let'stry the web site to verify that the web site is not functioning.
HOL-2101-05-CMP
Page 16HOL-2101-05-CMP
Moad Electronic Store Connection
Open a new tab in the Chrome browser for the Moad Application
1. Click the bookmark MOAD Electronic Store.2. Click the Chrome vRealize Operations tab.
Well this isn't good! Using vRealize Operations lets work through the problem.
Verify Startup is Running
1. NOTE: If the web page is still responding with the correct main page then go backto the Introduction to make sure the 2101-05-M1-STARTUP process is running.You should see a process running as shown in the image. If the Moad Application
is showing a error then continue with next page.
HOL-2101-05-CMP
Page 17HOL-2101-05-CMP
Back to Home Screen
1. Click Home if you are not at the Quick Start page.2. Click Virtual Machine Dashboard.
HOL-2101-05-CMP
Page 18HOL-2101-05-CMP
Troubleshoot a VM Dashboard
Notice this dashboard is broken into widgets which are numbered. The first widget isselecting the virtual machine we are concerned about.
1. Click Moad-Mysql in widget #1 to update the dashboard for that VM.2. Notice there are alerts in widget #3. The most critical is the CPU Usage. Lets
take a look at additional information on this dashboard.
Note: In your lab environment it takes a couple minutes for "_App Server CPU Usage" toappear as a active alert.
3. Click the Refresh button after a minute to update the dashboard. You may haveto click refresh a couple times but you will get the CPU usage alert within acouple cycles.
4. Scroll down the dashboard to review the other details on this dashboard.
HOL-2101-05-CMP
Page 19HOL-2101-05-CMP
Dashboard Stress Components
1. In widget #6 you find there has been a spike in CPU usage.2. Let's scroll back to the alert widget.
MySQL Server CPU Alerts
1. Click _App Server CPU Usage alert to look at the details of the alert.
HOL-2101-05-CMP
Page 20HOL-2101-05-CMP
Alert Details
1. The alert was triggered because the CPU usage exceeded 50%. Next is toreview the processes and see what is consuming the CPU.
2. There are 2 recommendations for this alert. The first contains an action to addmore CPU to the Moad-MySQL VM. Rather than just add more CPU, let's identifythe root cause for the sudden spike in CPU usage.
3. Click the > to view the other recommendation.
HOL-2101-05-CMP
Page 21HOL-2101-05-CMP
Recommendation #2
1. The second recommendation is to review the processes which are consuming theCPU. Also if needed is the command to kill processes.
2. Click the Metrics tab.
HOL-2101-05-CMP
Page 22HOL-2101-05-CMP
All Metrics Chart
The metric charts are helpful to view the metrics that vROps has collected and thetimeframe the issue has existed.
1. Enter demand in the filter field.2. Double-Click CPU Demand (%) to add the metric to the chart view. You may
have to scroll down the metric list to the Utilization group. NOTE: Make sure toDouble-click the metric or nothing will appear in the chart.
3. Click the Calendar icon4. Click the Last Hour to for better detail.5. Click GO to set the chart timeframe. Now the chart shows the CPU demand for
the last hour. We can see that a sudden spike in CPU has occurred. Let's addmore CPU to the VM to see if that fixes the alert.
HOL-2101-05-CMP
Page 23HOL-2101-05-CMP
Add CPU Action
1. Click Actions for a list of actions that are enabled for virtual machines.2. Click Set CPU Count for VM.
HOL-2101-05-CMP
Page 24HOL-2101-05-CMP
Set CPU Count
1. Enter 2 in the New CPU field. Note: Hot Add has been enabled for this VM so the"Power off Allowed" is not necessary for this change.
2. Click BEGIN ACTION
Task Started
1. Click Recent Tasks to check that the task is completed.
HOL-2101-05-CMP
Page 25HOL-2101-05-CMP
Recent Tasks
You can see in the Status column the task has completed. You may have to Refresh thescreen to see the completed state.
Add a new Chrome browser tab and login to vCenter and see the task has beencompleted as well.
HOL-2101-05-CMP
Page 26HOL-2101-05-CMP
Login to vCenter
1. The User Name and Password should already be set but if needed
User Name is: [email protected] and password is VMware1!
2. Click Login.
HOL-2101-05-CMP
Page 27HOL-2101-05-CMP
vCenter
1. Click Moad-MySQL after expanding the object tree as shown.2. Scroll if needed to view the VM Hardware section.3. Notice the number of CPU(s) is set to 2.
Click the vRops tab in the browser to return to vRealize Operations.
Recent Tasks
1. Click Moad-MySQL to return to the details page.
HOL-2101-05-CMP
Page 28HOL-2101-05-CMP
Return to Moad-MySQL Metrics
1. Click Metrics. Note: If the back button is unavailable you can also click theSummary tab for Moad-MySQL in the description field to return to thesummary page as well.
2. Enter Demand to filter the metrics list.3. Double-Click CPU Demand (%)4. Click the Calendar icon5. Change the time frame to Last Hour as completed earlier in the module.6. Click GO to update the chart.
You should see in the chart the CPU has dropped from the 100% since setting the CPU to2. The CPU usage has lowered but still indicates there is a process consuming CPU andit isn't the MySQL Service because it is stopped. Let's use the Get Top Processes Actionto see what process is consuming the CPU.
HOL-2101-05-CMP
Page 29HOL-2101-05-CMP
Select Actions
1. Click Actions for the Moad-MySQL server.
2. Click Get Top Processes to view the process list.
HOL-2101-05-CMP
Page 30HOL-2101-05-CMP
Run Process List
1. Click RUN to retrieve the top 10 processes
HOL-2101-05-CMP
Page 31HOL-2101-05-CMP
Top Processes
1. Notice the top processes are bzip2 and dd. They are consuming over 50% of theCPU. These are rogue processes that need to be stopped. Also you notice themysqld service is not running and this service is needed for the database of theMoad application. Let's start a putty session to kill the bzip2, dd and restart themysqld service.
2. Click Refresh if the 2 processes (bzip2 and dd) are not listed.
Start Putty
HOL-2101-05-CMP
Page 32HOL-2101-05-CMP
1. Double-Click the putty icon on the bottom of the screen to start the program
Putty to Moad-MySQL
1. Double-click Moad-SQL in the list of hosts to start the putty session. You willautomatically be logged in as the root user.
HOL-2101-05-CMP
Page 33HOL-2101-05-CMP
Stop and Start Services
1. Enter pkill -x bzip2 to stop the process consuming the CPU. The dd process willalso be killed automatically.
2. Enter systemctl start mysql to start the database service.
Switch back to the vROps browser tab and refresh the services list for Moad-MySQL
HOL-2101-05-CMP
Page 34HOL-2101-05-CMP
Refresh Top Processes
1. Enter 50 for the number of processes to list. This is because the mysqld serviceCPU usage is very low currently.
2. Click REFRESH to update the list of processes.3. The first thing you notice is the bzip2 and the dd processes are no longer running
on the host. Scroll down the list and you will find the mysqld process is runningindicating the database is up and running. Lets check the application web page.
4. Click X to close the window.
HOL-2101-05-CMP
Page 35HOL-2101-05-CMP
Refresh Moad-MySQL Metric Chart
1. Click the REFRESH ICON to update the chart. The chart is updated everyminute. You can see the changes every minute by continuing to use the Refreshicon every minute.
2. Notice the CPU % is dropping and will get to a steady state of less than 10%3. Click the Summary tab
NOTE: In the lab environment it can take up to 4 - 5 minutes for the chart to update asshown in the image.
HOL-2101-05-CMP
Page 36HOL-2101-05-CMP
Moad-MySQL Summary
1. Notice the _App Server CPU Usage Alert no longer exists because it wasautomatically cleared. There is a snapshot alert but we are not concerned aboutthat alert for now.
NOTE: In the lab environment it can take up to 4 minutes for the alert to clear asshown in the image. The browser Refresh icon can help update the Summary screen ifyou don't see the alert disappear.
HOL-2101-05-CMP
Page 37HOL-2101-05-CMP
The Moad Application Web Page
Let's view the contents of our application and try adding a new record.
1. Click the Moad Electronic Store bookmark or refresh the browser tab if youstill have the Moad Application tab opened.
Success! Now you can go shopping.
End of Lesson.
1. Close the Chrome browser.2. Close the Moad-MySQL Putty session.3. Close the 2101-05-M1-STARTUP command window if still opened.
HOL-2101-05-CMP
Page 38HOL-2101-05-CMP
ConclusionThis Module contained the following lessons:
• How to diagnose performance using the Virtual Machine troubleshootingdashboard
• Understand alerts and the symptoms indicating a issue is occurring.• Use Actions to set the number of CPU's, Get Top processes• Fix the application
You've finished module 1
Congratulations on completing the lab module.
If you are looking for additional general information on (vROPs) vRealize Operations 8.1,try one of these:
• VMware Product Public Page - vRealize Operations:https://www.vmware.com/products/vrealize-operations.html
• vRealize Operations 8.1 - Release Notes: https://docs.vmware.com/en/vRealize-Operations-Manager/8.1/rn/vRealize-Operations-Manager-81.html
HOL-2101-05-CMP
Page 39HOL-2101-05-CMP
• vRealize Operations 8.1 - Documentation: https://docs.vmware.com/en/vRealize-Operations-Manager/8.1/com.vmware.vcom.core.doc/GUID-7E6B5805-3D2F-41C4-ADFF-B7248386E7AC.html
• VMware Cloud Management Blog - What's New in vRealize Operations8.1: https://blogs.vmware.com/management/2020/03/whats-new-in-vrealize-operations-8-1.html
Proceed to any module below which interests you most.
• Module 2 - Utilizing vRealize Log Insight to find issues in ourenvironment (45 minutes) (Advanced) Determine what cost savings exist, findwaste, and perform automated reclamation
• Module 3 - Your application team is asking what vRealize Operations cando to help monitor their application (45 minutes) (Advanced) Discover thenew and improved What-if planning scenarios and the ability to run concurrentscenarios
• Module 4 - Determine the health of our vRealize Operations environment(30 minutes) (Advanced) Explore options to view the time remaining for a givenCluster based on available capacity of specific resources. Also learn about optionsto model the number of VMs that might fit into the remaining capacity
• Module 5 - Using vRealize Operations for vSAN dashboards to evaluateand manage storage (30 minutes) (Advanced) Learn about OperationsManagers various costing features and functionality
From here you can:
1. Click to advanced to the next page and continue with the next lab module2. Open the TABLE OF CONTENTS to jump to any module or lesson in this lab
manual3. Click on the END button if you are done with the lab for now and want to exit
HOL-2101-05-CMP
Page 40HOL-2101-05-CMP
Module 2 - UtilizingvRealize Log Insight to
find issues in ourenvironment (45 minutes)
HOL-2101-05-CMP
Page 41HOL-2101-05-CMP
IntroductionThis module will introduce you to using the Troubleshooting Workbench. This toolintelligently gathers and correlates object metric data including log data, which allowsfor quicker troubleshooting. It provides a single view of potential evidence and rootcause analysis. You can adjust time and topology scope to find the problem, pininteresting metrics, dismiss others, and use Log Insight for additional evidence.
HOL-2101-05-CMP
Page 42HOL-2101-05-CMP
Log in to the vRealize OperationsThis lab environment is running two different instances of vRealize Operations. We havethe different vRealize Operations instances for different use cases. The lab instancesare:
• Live Instance: Connected to the small running vSphere environment in the lab.There isn't a large inventory of objects in this instance but it allows us to interactwith a live system including vCenter.
• Historical View Mode (HVM) Instance: Running a time loop of data that wascaptured in the past. This instance has a much larger inventory of objects but,since it is not currently connected to a vCenter Server, we cannot perform anyactions here.
In this lesson we will be using the live Instance of vRealize Operations.
If you are not currently logged into any instance of vRealize Operations, continue to thenext page, but if you are already logged into the live (not historical) instance ofvRealize Operations, click to skip ahead.
Open the Chrome Browser from Windows Quick LaunchTask Bar
If your browser isn't already open, launch Google Chrome
1. Click the Chrome icon on the Windows Quick Launch Task Bar
The browser Bookmarks Bar has links to the different instances of vRealize Operationsthat are running in the lab.
1. The Live instance of vRealize Operations has already been started for you.
HOL-2101-05-CMP
Page 43HOL-2101-05-CMP
Log in to vRealize Operations
vRealize Operations is integrated with VMware Identity Manager which we will use foruser authentication in this lab.
VMware Identity Manager should be pre-selected as the identity source.
1. Click the drop-down arrow if VMware Identity Manager is not selected.2. Click REDIRECT to be taken to the authentication page.
HOL-2101-05-CMP
Page 44HOL-2101-05-CMP
VMware Identity Manager Login
For the Live instance of vRealize Operations instance, the default username andpassword should already be entered. The username and password are listed if you needto enter them.
username: holadmin
password: VMware1!
1. Click Sign in
HOL-2101-05-CMP
Page 45HOL-2101-05-CMP
vRealize Operations Home Screen
You should be at the vRealize Operations Home screen and ready to start the module.
HOL-2101-05-CMP
Page 46HOL-2101-05-CMP
vRealize Operations and vRealize LogInsight Better TogetherFor this lab you will be using the Moad Electronic Store App that consists of 2 virtualmachines. The use case for this lesson demonstrates how to use vRealize Operationsworkbench combined with Log Insight when a vmnic2 connected to a host goes down.
Moad Electronic Store
HOL-2101-05-CMP
Page 47HOL-2101-05-CMP
Now we'll initiate a network fault with the host esx-01a.corp.local. The network issuewill affect the Moad application that is running. There are 2 virtual machines (Moad-Weband Moad-MySQL) which run the application.
Check Moad Application
1. Click the + to add a new browser tab2. Click the MOAD Electronic Store bookmark to View the current status of the
application.3. Click the vRealize Operations tab to return to the Home screen.
HOL-2101-05-CMP
Page 48HOL-2101-05-CMP
Troubleshoot Host Dashboard
1. Click Host located in the Troubleshooting section to review the current status ofesx-01a.
HOL-2101-05-CMP
Page 49HOL-2101-05-CMP
Host esx-01a
1. Click esx-01a.corp.local to set the focus of the dashboard to that host.2. Observe there are no current alerts in widget #4.3. Widget #6 shows the relatives for esx-01a.corp.local. You will notice that Moad-
MySQL is running on this host.
So let's force an alert to trigger by downing vmnic2 on esx-01a which is connected to adistributed switch called Database-VDS-Comp. The virtual machine Moad-MySQL is alsoconnected to that switch.
HOL-2101-05-CMP
Page 50HOL-2101-05-CMP
Putty to esx-01a
1. Click the putty icon on the toolbar at the bottom of the desktop.2. Double-click esx-01a.corp.local to start a putty session for our host.
HOL-2101-05-CMP
Page 51HOL-2101-05-CMP
List vmnic Status using ESXCLi
1. Click Maximize to expand the putty window. It makes it easier to read the resultsof the next step.
2. Type esxcli network nic list and hit ENTER to see the status of the vmnic's onesx-01a.
3. Observe that vmnic2 is in a Up state.
Minimize the putty session because you will need it in later steps.
Execute Down network
HOL-2101-05-CMP
Page 52HOL-2101-05-CMP
1. Double-click the Lab Files folder on the desktop2. Double-click the 2101-01-M2-Down-network.bat to execute. A shell window
will briefly appear and then disappear when the batch file is executed. Let's takea look at the esx-01a host.
Open the minimized putty session.
List vmnic Status using ESXCLi
1. Type esxcli network nic list and hit ENTER to see the status of the vmnic's onesx-01a.
2. You should see vmnic2 is currently down. This is a result of the2101-01-M2-Down-network.bat file that was run earlier.
Let's switch back to the Chrome tab for the Moad Application.
HOL-2101-05-CMP
Page 53HOL-2101-05-CMP
Moad Application
1. Click the Moad Application tab in the browser.2. Click the browser Refresh icon. It takes about 10-15 seconds before the ERROR
occurs. The result is Moad-Web is not able to connect to Moad-MySQL VMbecause vmnic2 is in a down state which is the network used for the Moad VM'sto communicate.
Let's switch back to the vRealize Operations tab.
Refresh vRealize Operations
1. Return to the vRealize Operations tab and click the browser Refresh icon toupdate the dashboard.
HOL-2101-05-CMP
Page 54HOL-2101-05-CMP
Troubleshooting Host Dashboard
1. Notice widget #6 the health status for esx-01a.corp.local is red indicating aissue with the host. Also there is an active alert for the host.
2. Click on the alert The host has lost connectivity to a dvPort to see moredetails.
HOL-2101-05-CMP
Page 55HOL-2101-05-CMP
Alert Details
From the alert details page, we can understand how the alert is defined and what alertsymptoms were triggered when vmnic2 was brought down. To understand more of themetrics that potentially are affected, we need to look at the potential evidence.
1. Click Potential Evidence
HOL-2101-05-CMP
Page 56HOL-2101-05-CMP
Potential Evidence
The Potential Evidence screen is a collection of events, property changes andanomalous metrics that potentially are relevant to a given alert.
NOTE: Your results may vary from this screenshot depending on the time range andscope currently set in the workbench. You may not have any evidence shown becauseof the time range. This is because the lab environments have minimal history.
1. Click LAUNCH WORKBENCH to investigate the evidence.
HOL-2101-05-CMP
Page 57HOL-2101-05-CMP
Set Time to Last Hour
1. Click the Range dropdown as shown in the image.2. Click Last Hour.3. Click GO to set the time for collected evidence.
HOL-2101-05-CMP
Page 58HOL-2101-05-CMP
Using the Troubleshooting Workbench
The troubleshooting workbench provides potential pieces of evidence of a problemwithin a specific scope and time range. The Selected Scope control on the left of theTroubleshooting Workbench page allows you to vary the scope, such as selecting onlythe object that you are investigating, or increasing the scope to include severalupstream and downstream relationships.
1. Click the + to increase the scope.2. Click the All Objects dropdown and observe the different types of objects you
can choose for the scope.
As you increase the scope, additional evidence is presented under Events, PropertyChanges, and Metrics. As you increase the scope, more objects are displayed in theinventory tree. Also as mentioned earlier in the lab, you can use the custom scope tochange the focus of the workbench.
HOL-2101-05-CMP
Page 59HOL-2101-05-CMP
Increased Scope to Datacenter
1. Notice that the increase in scope has added datacenter events. In this use casethe event "Lost network connectivity on DVPorts" is the same at the host, clusterand datacenter level. Depending on the use case, there will be different eventsoccurring within the environment. The troubleshooting workbench brings all thisinformation to one spot.
HOL-2101-05-CMP
Page 60HOL-2101-05-CMP
Events, Property Changes and Metrics
There are 3 main areas in the workbench:
1. Events
Displays events, based on a change in the metrics. Events for metrics that havebreached the usual behavior and major events that have occurred within the selectedscope and time are displayed. The cards are based on dynamic thresholds for a metric,which is calculated based on historical and incoming data.
2. Property Changes
Displays important configuration changes that occurred within the selected scope andtime. Both single and multiple property changes are displayed. For multiple propertychanges, you can view the latest and previous changes.
3. Anomalous Metrics
Metrics which have shown drastic changes within the selected scope and time. Ranksthe results based on the degree of change. The most recent anomalous metric based ona time-sliced comparison in the current time range is given the highest weightage.
When you pin a metric, it appears in the Metrics tab of the Troubleshooting Workbench.You can perform further investigation on the metric in the Metrics tab. You can
HOL-2101-05-CMP
Page 61HOL-2101-05-CMP
compared the pinned metrics with other metrics displayed in the tab. You can close thepinned metrics and browse other metrics for specific objects.
Login to Log Insight
You also have the ability to launch Log Insight in context from the Troubleshootingworkbench. This will automatically create a filter in Log Insight for esx-01a.corp.localand will only shows logs for that host.
1. Click Logs2. Click VMware Identity Manager from the drop down menu if it is not already
selected.3. Click LOGIN VIA SSO
HOL-2101-05-CMP
Page 62HOL-2101-05-CMP
Log Insight Logs
Because you clicked Logs from the esx-01a workbench, a filter was automaticallycreated so only esx-01a logs are displayed. We are only interested in logs pertaining tothe down network so let's add an additional filter.
1. Click ADD FILTER
Add Text Filter for VMnic Errors
HOL-2101-05-CMP
Page 63HOL-2101-05-CMP
1. Check that text is selected.2. Click does not contain and change to contains as shown.3. Type vmnic2 to filter the messages to those that include vmnic2 and esx-01a.4. Click the Search icon to filter the messages.5. Notice there are logs pertaining to vmnic2 being down and lost connectivity.6. Click Potential Evidence to return to the workbench page.
Let's bring the vmnic2 interface up.
Restart VMnic2
1. Double-click the Lab Files folder on the desktop.2. Double-click the 2101-01-M2-Up-network.bat to execute the script and restart
the VMnic2.
HOL-2101-05-CMP
Page 64HOL-2101-05-CMP
ESXcli List
1. Return to the putty session for esx-01a and type esxcli network nic list to see thecurrent status of the nic's. As shown in the image vmnic2 is now in an up state.
2. Minimize the putty session and switch back to the browser.
Moad Application
1. Click the Moad Application tab.2. Click the browser refresh icon a few times to create some network traffic
between Moad-Web and Moad-MySQL The web server can now display thecontents of the database server.
3. Click the vRealize Operations tab in the browser.
HOL-2101-05-CMP
Page 65HOL-2101-05-CMP
vRealize Operations
1. You may see that esx-01a.corp.local is still showing Unhealthy Redindicating the dashboard has not been updated.
2. Click the browser Refresh
HOL-2101-05-CMP
Page 66HOL-2101-05-CMP
Healthy Host
1. Now the host is showing Green indicating it is in a healthy state and the Networkdown alert has been automatically cleared.
End of Module
To Clean up the environment for the next module:
• Close the Putty sesssion• Close the Chrome browser
HOL-2101-05-CMP
Page 67HOL-2101-05-CMP
ConclusionIn this module you learned about:
• Using the Host Troubleshooting Dashboard and the Troubleshooting Workbench todiagnose a network issue.
• Reviewed logs from Log Insight• Check the current status of the application once connectivity has been restored.
You've finished module 2
Congratulations on completing module 2.
If you are looking for additional information on vRealize Operations, you can start here:https://www.vmware.com/products/vrealize-suite.html
You may proceed to the next module by advancing to the next page. If you want to jumpto a particular module, follow one of the links below.
• Module 1 - Why is My VM Not Performing Well? How Do I Fix It? (30minutes)
• Module 3 - In-Guest Monitoring of the Operating System andApplications (45 minutes)
• Module 4 - Assess Your vSphere Configuration For Compliance WithIndustry or Custom Standards (30 minutes)
• Module 5 - Monitoring and Planning your vSAN Environment withvRealize Operations (15 minutes)
Or if you want to end your lab,
1. Click on the END button at the top of the page.
HOL-2101-05-CMP
Page 68HOL-2101-05-CMP
Module 3 - Yourapplication team is askingwhat vRealize Operations
can do to help monitortheir application (45
minutes)
HOL-2101-05-CMP
Page 69HOL-2101-05-CMP
IntroductionIn this module, we are going to immerse ourselves in a scenario where theInfrastructure Administrator at Rainpole Company receives a report from an ApplicationOwner that a website is not performing like it should. The Administrator will utilizevRealize Operations' new agent-based application on OS discovery solution to determinewhere the application is affected and correlate that back to infrastructure to lower themean time to resolution (MTTR) for the organization. We will go through the completeprocess of discovering the applications, listing the applications, and troubleshooting thesituation.
HOL-2101-05-CMP
Page 70HOL-2101-05-CMP
Hands-on Labs Interactive Simulation:Your application team is asking whatvRealize Operations can do to helpmonitor their applicationThis part of the lab is presented as a Hands-on Labs Interactive Simulation. This willallow you to experience steps which are too time-consuming or resource intensive to dolive in the lab environment. In this simulation, you can use the software interface as ifyou are interacting with a live environment.
1. Click here to open the interactive simulation. It will open in a new browserwindow or tab.
2. When finished, click the “Return to the lab” link to continue with this lab.
The lab continues to run in the background. If the lab goes into standby mode, you canresume it after completing the module.
HOL-2101-05-CMP
Page 71HOL-2101-05-CMP
ConclusionIn this module, we explored the new monitoring capabilities of vRealize Operations. Wetook you through how to troubleshoot an application by assessing the underlyinginfrastructure after the app owner experienced issues.
You've finished the module
Congratulations on completing the lab module.
If you are looking for additional information, you can start by viewing the vRealizeOperations product page.
From here you can:
1. Click to advance to the next page and continue with the next lab module2. Open the TABLE OF CONTENTS to jump to any module or lesson in this lab
manual3. Click on the END button if you are done with the lab for now and want to exit
HOL-2101-05-CMP
Page 72HOL-2101-05-CMP
Module 4 - Determine thehealth of our vRealize
Operations environment(30 minutes)
HOL-2101-05-CMP
Page 73HOL-2101-05-CMP
IntroductionIn this module, we will use vRealize Operations to monitor vRealize Operations itself.Typically, we use vRealize Operations to ensure our systems are healthy, but we alsoneed to make sure we are taking care of vRealize Operations. We will walk through thebasics of where to look for help and ensure our management stack is healthy andefficient.
HOL-2101-05-CMP
Page 74HOL-2101-05-CMP
Log in to the vRealize Operations HVMInstanceThis lab environment is running two different instances of vRealize Operations and oneinstance of vRealize Log Insight. We have the different vRealize Operations instances inorder to be able to work through different use cases that have unique requirements. Thelab vRealize Operations instances are as follows:
• Live Instance: Connected to the small running vSphere environment in the lab.There isn't a large inventory of objects in this instance but it allows us to interactwith vCenter Server.
• Historical View Mode (HVM) Instance: Running a time loop of data that wascaptured in the past. This instance has a much larger inventory of objects butsince it is not currently connected to a vCenter Server, we cannot perform anyactions here.
In this lesson we will be using the HVM instance of vRealize Operations.
If you are already logged into the HVM (not live) instance of vRealize Operations, clickhere to skip ahead.
Open the Chrome Browser from Windows Quick LaunchTask Bar
If your browser isn't already open, launch Google Chrome
1. Click the Chrome icon on the Windows Quick Launch Task Bar
The browser home page has links to the different instances of vRealize Operations thatare running in the lab.
1. Click the vrops-HVM link on the toolbar, to open the vRealize Operations UI.
HOL-2101-05-CMP
Page 75HOL-2101-05-CMP
Log in to vRealize Operations
The user and password information should already be filled in. However, if needed, justtype it in.
username: admin
password: VMware1!
1. Click LOG IN
HOL-2101-05-CMP
Page 76HOL-2101-05-CMP
Using Dashboards to Monitor HealthIn this lesson, we will learn how to use dashboards within vRealize Operations to gainvisibility on vRealize Operations performance, self-health, and troubleshooting. This isthe first step to ensuring that vRealize Operations, our go-to solution for overall ITmonitoring and health, is itself healthy. We will also learn how to create custom alertsbased on a particular metric straight from our dashboard view.
Access All Dashboards
We should now be in the Home screen after logging into the HVM instance. We will nownavigate to view all the Dashboards we have available to use in order to determine theappropriate one to assess vRealize Operations health.
1. Click on the button to minimize the side navigation bar at the top-left corner2. Click on Dashboards
HOL-2101-05-CMP
Page 77HOL-2101-05-CMP
Find Self Performance Details Dashboard
Now we want to determine which dashboard we want to use from the inventory we haveavailable. We want to narrow down our search to focus on dashboards related tovRealize Operations.
1. Click on DASHBOARDS in blue to expand the dashboards inventory list2. Type 'operations' in the Search for a dashboard filter3. Click on vRealize Operations to view all dashboards related to vRealize
Operations4. Click on Self Performance Details
Monitor vRealize Operations Performance
The Self Performing Details dashboard has several views into performance metrics ofyour vRealize Operations node. We will walk through some metrics that are worthhighlighting within this dashboard.
HOL-2101-05-CMP
Page 78HOL-2101-05-CMP
NOTE: The resolution may not render everything properly and as displayed in the image.You may have to scroll for a while to see all the widgets displayed.
1. Verify CPU usage, memory used, memory remaining, and other high-level metrics2. Verify utilization for disk partitions associated with the vRealize Operations nodes
View CPU and Memory Metrics
We will now verify historical trends of CPU and memory for several components of thevRealize Operations node. The orange data points that appear on the individual chartsrepresent the minimum and maximum values of the respective metric. We will thenexplore the Analytics CPU chart further.
1. Scroll down to the CPU/Memory view2. Double-click on any area within the Analytics CPU line chart
Explore CPU Trends and Anomalies
We can now more clearly see the historical CPU trend for vRealize Operations Analytics.Within this view, we can get more details on Analytics CPU by narrowing down the datato a particular date range and by adding trendlines and anomalies. Let's explore whatwe can do to customize our view of this metric.
HOL-2101-05-CMP
Page 79HOL-2101-05-CMP
1. Click on the Anomalies button to see what is flagged as abnormal givenhistorical trends
2. Ensure the Zoom the view button is pressed3. With your mouse, highlight a portion of the graph by leaving the mouse clicked
for a range of the data, which will adjust the graph to show you just the period oftime you have highlighted.
4. Click on the Calendar icon drop-down to change the range of time of data beingshown, from within the last hour to within the last year. Feel free to play aroundwith the different date ranges in this view.
After going through the steps above, we notice that there are no anomalies flagged byvRealize Operations. Although the chart may suggest that CPU spiked abnormally,vRealize Operations determines whether that CPU change is normal based on how andwhen CPU has behaved in the past in order to reduce false positive flags.
Create Custom Alerts from Dashboard
What if we want to know when CPU (or any other resource/metric) has reached a certainthreshold, even though it is not flagged as an anomaly? We can create a custom alert!Let's walk through how we can create our own alert right from the chart we are viewing.
1. Click on the three-lined icon on the top-right part of the chart2. Click on Create an Alert Definition
HOL-2101-05-CMP
Page 80HOL-2101-05-CMP
Create Custom Alert
We will walk through a scenario where we want to be notified when CPU usage reachesa certain percentage. We can customize the criticality of our alert so that we can bemore targeted when troubleshooting critical alerts.
1. Create a Name for the alert by typing 'vROps Analytics CPU usage above 50%'2. Click on the Condition dropdown and select is greater than3. Enter '50' in the Value field4. Click on the Criticality dropdown and select Warning5. Click CREATE
HOL-2101-05-CMP
Page 81HOL-2101-05-CMP
Go Back to Dashboard
1. Click on the X at the top-right to exit the screen
Lesson Completed
You have finished this lesson on using the vRealize Operations Self Performance Detailsdashboard to monitor node health. Please continue to the next page to learn aboutcreating more custom alerts for your vRealize Operations environment.
HOL-2101-05-CMP
Page 82HOL-2101-05-CMP
Using Alerts for vRealize OperationsHealthIn this lesson, we will learn how to find and view alerts that have been triggered for ourvRealize Operations node. This is an important piece to begin troubleshooting vRealizeOperations when something goes wrong. We will also walk through how to create ourown custom alert based on certain health symptoms.
Go to All Alerts
1. Ensure the side navigation menu is expanded by clicking on >>2. Click on Alerts from the top menu
View Alerts and Symptoms for vRealize Operations
In the Alerts page, we first land on a report with all active alerts. Alerts within vRealizeOperations are triggered by a set of defined symptoms - that can also be customizedand defined - and provide a set of recommendations to address those symptoms. Let'sfirst determine what symptoms we have defined for our vRealize Operations node healthand performance.
1. Click on Symptom Definitions from the left-hand menu2. On the Quick filter (Alert) field, type 'vRealize Operations'
HOL-2101-05-CMP
Page 83HOL-2101-05-CMP
The result page will contain all symptoms tied to the vRealize Operations node. Let'sdive into one of these symptoms.
Symptom Definition Details
1. Click on the second alert called vRealize Operations Analytics service crashmultiple times by clicking on the text
2. Verify the condition against a certain metric that defines the symptom as well asthe criticality of the symptom
3. Click on the Used In Alert Definitions dropdown to verify how many and whichalerts are using this particular symptom definition
4. Click on the Policies Enabled dropdown to verify which policies are enabled withthis symptom
Create Alerts
HOL-2101-05-CMP
Page 84HOL-2101-05-CMP
Now that we know how symptoms are associated with alerts, we can walk through howto create your own alert based on a couple of pre-defined symptoms.
1. Click on Alert Definitions from the left-hand menu2. Click on ADD
Alert Definition
1. Write a custom name (for example: 'My Alert') for your alert in the Name field2. Type 'vRealize Operations Cluster' in the Base Object Type search field3. Select vRealize Operations Cluster under 'vRealize Operations Adapter'4. Click NEXT
HOL-2101-05-CMP
Page 85HOL-2101-05-CMP
Add Symptoms to the Alert
We will now add symptoms that are already defined in our environment to the alert weare creating. We will focus on the Health and Risk of our cluster.
1. Drag the Average Health of group members is critically low SymptomName to the box
2. Drag the Average Risk of group members is critically high Symptom Nameto the box
3. Click on NEXT
Add Recommended Remediation Actions
We will skip adding Recommendations to our alert in this module.
1. Click on NEXT
HOL-2101-05-CMP
Page 86HOL-2101-05-CMP
Add Policies to Alert
Now we will enable policies to run if the vRealize Operations Cluster has the Health andRisk symptoms that we assigned to this alert.
1. Check the box next to vSphere Solution's Default Policy2. Select CANCEL
For the purposes of this lab, we have canceled this alert creation because theenvironment will not add any more alerts. However, we wanted to walk you through thesteps needed to create an alert for vRealize Operations health.
Lesson Completed
You have finished this lesson on finding, viewing, and creating custom alerts for yourvRealize Operations node. Please continue to the next page to conclude this module.
HOL-2101-05-CMP
Page 87HOL-2101-05-CMP
ConclusionIn this module, we learned how to use vRealize Operations dashboards to monitor self-health metrics and how to create alerts associated with vRealize Operations healthmetrics. With these basic tools, we can ensure vRealize Operations is as healthy aspossible to continue being our reliable infrastructure monitoring and managementsolution.
You've finished the module
Congratulations on completing the lab module.
If you are looking for additional information, below are a few links with availableresources that we were not able to cover in this module:
• Health Monitoring Management Pack for vRealize Suite stack• SDDC Health Overview Dashboard in vRealize Operations
From here you can:
1. Click to advance to the next page and continue with the next lab module2. Open the TABLE OF CONTENTS to jump to any module or lesson in this lab
manual
HOL-2101-05-CMP
Page 88HOL-2101-05-CMP
3. Click on the END button if you are done with the lab for now and want to exit
HOL-2101-05-CMP
Page 89HOL-2101-05-CMP
Module 5 - Using vRealizeOperations for vSAN
dashboards to evaluateand manage storage (30
minutes)
HOL-2101-05-CMP
Page 90HOL-2101-05-CMP
IntroductionIn this module, we will learn how to use vRealize Operations to gain visibility into everyvSAN cluster in your environment in order to monitor and troubleshoot all aspects of ourvSAN clusters. We will also discuss how to run modeling scenarios to determine capacityimplications and costs associated with growing or shrinking your hyperconvergedfootprint. To have more accurate cost data available, we will learn how to add our actualcosts for our hyperconverged infrastructure, which feed into all dashboards withinvRealize Operations.
HOL-2101-05-CMP
Page 91HOL-2101-05-CMP
Log in to the vRealize Operations HVMInstanceThis lab environment is running two different instances of vRealize Operations and oneinstance of vRealize Log Insight. We have the different vRealize Operations instances inorder to be able to work through different use cases that have unique requirements. Thelab vRealize Operations instances are as follows:
• Live Instance: Connected to the small running vSphere environment in the lab.There isn't a large inventory of objects in this instance but it allows us to interactwith vCenter Server.
• Historical View Mode (HVM) Instance: Running a time loop of data that wascaptured in the past. This instance has a much larger inventory of objects butsince it is not currently connected to a vCenter Server, we cannot perform anyactions here.
In this lesson we will be using the HVM instance of vRealize Operations.
If you are already logged into the HVM (not live) instance of vRealize Operations, clickhere to skip ahead.
Open the Chrome Browser from Windows Quick LaunchTask Bar
If your browser isn't already open, launch Google Chrome
1. Click the Chrome icon on the Windows Quick Launch Task Bar
The browser home page has links to the different instances of vRealize Operations thatare running in the lab.
1. Click the vrops-HVM link on the toolbar, to open the vRealize Operations UI.
HOL-2101-05-CMP
Page 92HOL-2101-05-CMP
Log in to vRealize Operations
The user and password information should already be filled in. However, if needed, justtype it in.
username: admin
password: VMware1!
1. Click LOG IN
HOL-2101-05-CMP
Page 93HOL-2101-05-CMP
Troubleshooting vSANIn this lesson, we will learn how to use the Troubleshooting vSAN dashboard to assessand address vSAN health and performance.
Troubleshoot vSAN
We can access the Troubleshoot vSAN dashboard either via the Home menu or bysearching through Dashboards. We will access the dashboard directly in the Home page.
1. Under the Troubleshoot pillar, select vSAN
HOL-2101-05-CMP
Page 94HOL-2101-05-CMP
Alerts on vSAN Clusters
Let's say that we are seeing performance degradation on one of our clusters. Let's seewhat alerts have been triggered on this cluster to start deciphering the issue.
1. Hide the side navigation menu2. Select vSAN Cluster(Host Licensing Cluster 01) to filter by that cluster3. Select VMKernel NIC is experiencing high number of dropped packets
from the alerts widget
NOTE: Depending on the zoom settigs on your Chrome browser window, the widgetsmay appear in different parts of the screen/dashboard. You may need to scroll down/upto see the appropriate widgets.
Details on Alert
We can now see how many times this alert has been triggered on objects related to thevSAN Cluster we had selected. Let's find out more details on one of these alerts.
1. Select View Details for alert triggered on 'sc2-vesxi-linux-02.cmbu.local'
HOL-2101-05-CMP
Page 95HOL-2101-05-CMP
See Recommended Actions
We can see all symptoms on the host that triggered the alert and recommendations tofix the issue. You can also take action on the recommendation if you so desire.
1. Verify the Recommendation2. Select Actions from the Alert menu
From this menu, we will see all available actions for that specific alert. You can choose tocancel, suspend, and assign the alert as well as perform the recommended action onthe host directly within the vSphere Web Client. Taking action on the alert is out of scopefor this module. Let's go back to the Troubleshoot vSAN dashboard where we left off.
Take Action on Alerts
HOL-2101-05-CMP
Page 96HOL-2101-05-CMP
1. Expand the side navigation menu2. Select BACK
Determining Root of Health Issues
When we realize there is an issue with a specific object, it is often hard to determinewhat exactly is causing the issue. Is the object itself? Is it an issue with a related object?When trying to figure this out, we also often get a lot of push-back by different teamswho claim their resources are healthy.
For the remaining portion of this lesson, we will immerse ourselves in the scenariowhere different teams are banging on our door because they can't access the resourcesthey need. There is no consensus among infrastructure teams on what is causing theissue. We will now learn how to get to the bottom of health issues within our vSANclusters in order to speed up the troubleshooting process and remove the guesswork.We will also learn how to easily share our insights with these teams. Let's begin byexploring the health of our vSAN cluster's relatives.
vSAN Cluster and Relative Health
1. Hide the side navigation menu
HOL-2101-05-CMP
Page 97HOL-2101-05-CMP
2. Note that the vSAN Cluster object is healthy as it has no alerts3. Mouse over sc2-vesxi-linux-02.cmbu.local and note that it is the host that was
associated with the alert we just explored
The red badges on the relatives signal a critical alert with that particular object. Theyellow badge signals a less critical alert. This diagram helps us determine which objectswe need to tackle first to avoid associated issues with our vSAN cluster.
Other Visibility in Troubleshoot vSAN Dashboard
The Troubleshoot vSAN dashboard has several views into potential issues on yourstorage environment. Each view helps answer a lot of the questions that we would askourselves when troubleshooting vSAN. Other components we can view within thisdashboard include I/O read and write latency, CPU and memory demands from hosts,disk space utilization, network utilization, disk throughput, cache, disk group capacity,disk group usage balance, capacity disk latency, packet drops, and congestion.
You may browse through the different views within the dashboard to get a betterunderstanding of how the data is visualized. Since these are all meant for viewing, wewill not be showing you any further steps on this dashboard.
Sharing the Dashboard
When there is no consensus among infrastructure teams on the root of an issue, thebest way to clarify confusion is to show where the issue lies with data. Sharing adashboard is extremely useful when you want to collaborate with different teams as youtroubleshoot issues or monitor performance. In vRealize Operations, you can share adashboard either through a live link or by taking a static snapshot of your custom viewat the time of capture.
We will now walk through how to share a live dashboard. Further along in this module,we will walk through how to create a snapshot of a specific view that you can share withothers.
Start Sharing Live Dashboard
1. Select the three-dotted diagram on the top-right corner
HOL-2101-05-CMP
Page 98HOL-2101-05-CMP
Share Live URL
1. Ensure URL is selected2. Click on the Expiry drop-down3. Click on 3 Months4. Click on COPY LINK5. Click X to exit
Choosing these settings will allow people with this specific URL to access the dashboardonly for 3 months after the link is created. Therefore, there is flexibility in how long wewant shared dashboards to be accessed by team members.
Open New Incognito Window
1. Click on the three vertical dots on the top-right corner of the Chrome window2. Select New incognito window
HOL-2101-05-CMP
Page 99HOL-2101-05-CMP
Open Dashboard Shared Link
1. Right-click on the address bar2. Load the copied URL by selecting Paste and pressing Enter on the keyboard
We will now see the dashboard in read-only mode, which is what our link recipientswould see as well. They now have the visibility they need to determine with certaintywhere any potential issues lie and take action quickly.
Lesson Completed
This now completes the lesson on troubleshooting vSAN! Please continue to the nextpage to learn about assessing capacity needs for your vSAN environment.
HOL-2101-05-CMP
Page 100HOL-2101-05-CMP
vSAN Capacity ManagementIn this lesson, we will learn how to use a pre-built vSAN dashboard to explore vSANcluster utilization, disk usage, and potential resource reclamation opportunities to savecosts.
Go to Capacity Dashboard
1. Click on Dashboards2. Click on the DASHBOARDS dropdown3. Search for 'vsan' on the Search for dashboard filter4. Click on Capacity & Utilization5. Click on vSAN Capacity Overview dashboard
HOL-2101-05-CMP
Page 101HOL-2101-05-CMP
Explore vSAN cluster
In the vSAN Capacity Overview dashboard, we can dig into any particular cluster'shistorical capacity trends and forecasted time remaining of the resources. We willexplore information contained in the dashboard for a particular cluster that will unlockoptimization opportunities.
1. Select vSAN Cluster(sc2c01) from the List of vSAN Cluster(s)
Resource Usage Trends
It is important to understand how your CPU, memory, and disk have behaved altogetherin the past to have a better sense of where potential issues may lie. In this dashboard,we can see the high-level historical trends as well as dig into specific time frames. Let'swalk through how you can navigate into a particular point in time.
1. Navigate to the Cluster Utilization(%) widget, scrolling down as necessary
HOL-2101-05-CMP
Page 102HOL-2101-05-CMP
2. Ensure the Zoom the view magnifying glass is pressed3. Leaving the mouse pressed, highlight a particular area of the chart as shown to
update the date range
Customize Date Range
We can get very specific in the time-frames we want to visualize. Let's look at aparticular date range. Feel free to play around with the date ranges to see how the chartchanges accordingly.
1. Click on the calendar icon2. Click on Relative Date Range3. Enter 164. Select Days from the dropdown5. Select GO
HOL-2101-05-CMP
Page 103HOL-2101-05-CMP
Specify CPU, Memory, and Disk Utilization
We can also decide to include and/or exclude CPU, Memory, and Disk trends to drill intoparticular resource metrics. Let's look at a scenario where we only want to look at CPUand Disk.
1. Select Memory
We can play around with the metrics by clicking on each colored line in the legend inorder to include and exclude them from the chart.
Dedupe and Compression Savings
Let's walk through how we can assess the savings that vSAN is unlocking for us with itsdedupe and compression capabilities. Determining these savings can help us assesshow vSAN makes our infrastructure more efficient. We will also learn how we can sharethese with interested stakeholders.
High-Level Savings
1. Double-click on any part of the grayed-out area alongside the chart line under theDedupe / Compression Savings chart
HOL-2101-05-CMP
Page 104HOL-2101-05-CMP
Savings in a Date Range
Let's say that you implemented vSAN about a month ago and want to figure out yourefficiency savings in order to quantify the value of vSAN. Let's dig into our data for thelast month.
1. Select the calendar icon2. Select Last month from the Range dropdown3. Select GO
Share Savings Chart
We can share a static screenshot of this particular view as a PNG file with any interestedstakeholders. We could also share the complete live vSAN Capacity Overviewdashboard, but we will not walk through how to do that in this particular lesson.
1. Click the icon with three horizontal lines at the top right corner2. Select Save a full screen snapshot
HOL-2101-05-CMP
Page 105HOL-2101-05-CMP
Saving a Snapshot
In this environment, we will not be able to save our snapshot. We will skip saving oursnapshot for now.
1. Click Cancel
Back to Dashboard
1. Click the X to exit the view
Evaluate Disk Groups
HOL-2101-05-CMP
Page 106HOL-2101-05-CMP
1. Navigate to the Is disk usage balanced? widget and verify which disk groupsare balanced (green) and not balanced (red)
Note: Depending on zoom settings on your Chrome browser window, you may need toscroll up/down as necessary.
Evaluate Disk Space
1. Navigate to the Capacity Remaining widget and verify current disk capacityused and how disk capacity is remaining
2. Navigate to the Time Remaining widget and verify how long it will take until yourun out of disk space given historical trends
Note: Depending on zoom settings on your Chrome browser window, you may need toscroll up/down as necessary.
Storage Reclamation
HOL-2101-05-CMP
Page 107HOL-2101-05-CMP
1. Navigate to the Storage Reclaimation [sic] widget and note how much storagecapacity you can release by reclaiming unused resources
Note: Depending on zoom settings on your Chrome browser window, you may need toscroll up/down as necessary.
Knowing information about current utilization and future capacity needs, we are able tobetter plan our storage capacity and optimize our footprint.
Lesson Completed
We have completed this lesson on monitoring vSAN capacity! Please continue to thenext page to learn how to successfully plan your next hyperconverged infrastructureexpansion and consolidation.
HOL-2101-05-CMP
Page 108HOL-2101-05-CMP
Hands-on Labs Interactive Simulation:Using vRealize Operations for vSANdashboards to evaluate and managestorageThis part of the lab is presented as a Hands-on Labs Interactive Simulation. This willallow you to experience steps which are too time-consuming or resource intensive to dolive in the lab environment. In this simulation, you can use the software interface as ifyou are interacting with a live environment.
1. Click here to open the interactive simulation. It will open in a new browserwindow or tab.
2. When finished, click the “Return to the lab” link to continue with this lab.
The lab continues to run in the background. If the lab goes into standby mode, you canresume it after completing the module.
HOL-2101-05-CMP
Page 109HOL-2101-05-CMP
Customize Cost DriversIn this lesson, we will learn how to modify cost drivers in order to get accurate costingdata and reports based on our own hyperconverged infrastructure costs in vRealizeOperations.
Customize Cost Drivers
When going through the planning scenarios, a lot of people ask: "where do these costscome from"? The answer is that it comes from what are called cost drivers. vRealizeOperations has cost drivers pre-defined for compute, storage, and networkingcomponents based on industry-based averages; however, you can also update the costdrivers to match your actual costs for that hardware. Let's walk through how tocustomize the hyperconverged infrastructure costs.
Login to HVM Instance
If you are not already logged in to the vRealize Operations HVM instance from thebeginning of the module, go to this page for instructions on how to log in.
If you are logged in, you may proceed to the next page.
HOL-2101-05-CMP
Page 110HOL-2101-05-CMP
Access Cost Settings
1. Ensure the side menu navigation is expanded2. Select Administration from the top menu3. Click on the Configuration dropdown menu4. Select Cost Settings
Hyperconverged cost driver definition
HOL-2101-05-CMP
Page 111HOL-2101-05-CMP
1. Select Server Hardware : Hyper-Converged
Modify HCI Server Costs
We can modify cost drivers for all servers in our data center. Let's explore the firstserver on the list.
1. Select the first Cisco UCS server
Update Purchase Information
1. Select the Purchase Type dropdown menu2. Select Leased
HOL-2101-05-CMP
Page 112HOL-2101-05-CMP
3. Type 15000 under Cost Per Server4. Select SAVE5. Collapse the row
Lesson Completed
You have finished this lesson on customizing costing data! Please proceed to the nextpage to conclude the module.
HOL-2101-05-CMP
Page 113HOL-2101-05-CMP
ConclusionIn this module, we learned how to use vRealize Operations to gain visibility into everyvSAN cluster in your environment in order to monitor and troubleshoot all aspects of ourvSAN clusters. We used the vSAN Capacity Overview dashboard and the TroubleshootvSAN dashboard. We also discussed how to run modeling scenarios to determinecapacity implications and costs associated with growing and shrinking ourhyperconverged environment. To have more accurate cost data available, we learnedhow to add our actual costs for our hyperconverged infrastructure, which feed into alldashboards within vRealize Operations.
You've finished the module
Congratulations on completing the lab module.
If you are looking for additional information, you can start by viewing the vRealizeOperations product page.
From here you can:
1. Click to advance to the next page and continue with the next lab module2. Open the TABLE OF CONTENTS to jump to any module or lesson in this lab
manual
HOL-2101-05-CMP
Page 114HOL-2101-05-CMP
3. Click on the END button if you are done with the lab for now and want to exit
HOL-2101-05-CMP
Page 115HOL-2101-05-CMP
ConclusionThank you for participating in the VMware Hands-on Labs. Be sure to visithttp://hol.vmware.com/ to continue your lab experience online.
Lab SKU: HOL-2101-05-CMP
Version: 20200814-030141
HOL-2101-05-CMP
Page 116HOL-2101-05-CMP
Top Related