Release Notes for Cisco HX Data Platform, Release 4€¦ · •Cisco HyperFlex M5 Converged...
Transcript of Release Notes for Cisco HX Data Platform, Release 4€¦ · •Cisco HyperFlex M5 Converged...
Release Notes for Cisco HX Data PlatformRelease 40
First Published 2019-05-02
Last Modified 2020-05-11
IntroductionCisco HyperFlextrade Systems unlock the full potential of hyperconvergence The systems are based on anend-to-end software-defined infrastructure combining software-defined computing in the form of CiscoUnified Computing System (Cisco UCS) servers software-defined storage with the powerful Cisco HX DataPlatform and software-defined networking with the Cisco UCS fabric that integrates smoothly with CiscoApplication Centric Infrastructure (Cisco ACI) Together with a single point of connectivity and hardwaremanagement these technologies deliver a pre-integrated and adaptable cluster that is ready to provide a unifiedpool of resources to power applications as your business needs dictate
These release notes pertain to the Cisco HX Data Platform Release 40 and describe the features limitationsand caveats for the Cisco HX Data Platform
Revision History
DescriptionDateRelease
Added OTV for Stretched Clusterupdate in New Features for the40(2b) release
May 11 202040(2b)
Updated HX 40(2a) New Featuresection to show support for UCSMhardware and softwareenhancements and features fromCisco UCSM 41(1c) release
Updated Host Upgrade Utility(HUU) for M5 to UCS 40(4k) forHX 40(2a)
May 5 202040(2a)
Created release notes for Cisco HXData Platform Software Release40(2b)
April 22 202040(2b)
Updated M4 and M5Recommended FIServer Firmwareto UCS 40(4h) for HX 40(1b)
March 30 202040(1b)
Release Notes for Cisco HX Data Platform Release 401
DescriptionDateRelease
Updated M4 and M5Recommended FIServer Firmwareto UCS 40(4h) for HX 40(2a)
March 24 202040(2a)
Created release notes for Cisco HXData Platform Software Release40(2a)
February 11 202040(2a)
Updated release notes for deferredCisco HyperFlex Release HX35(2c)
January 15 202035(2c)
Updated M4 and M5Recommended FIServer Firmwareto UCS 40(4e) for HX 40(1b)40(1a) 35(2f) 35(2e) and35(2d)
December 23 201940(1b)
Added CSCvs28167 to the list ofResolved Caveats for HX 40(1b)and HX 40(1a)
December 13 201940(1b)
Added CSCvs02466 to the list ofOpen Caveats
November 25 201940(1b)
Updated info in the HyperFlexEdge and Firmware CompatibilityMatrix for 3x Deployments
Updated info in the Storage ClusterSpecifications
November 7 201940(1b)
Added CSCvj95606 andCSCvq24176 to the list of SecurityFixes
October 25 201940(1b)
Updated Recommended FIServerFirmware versions
October 8 201940(1b)
Updated HUUCIMC info in theHyperFlex Edge and FirmwareCompatibility Matrix for 4xDeployments
September 30 201940(1b)
AddedCSCvq41985 to new sectionfor Related Caveats
September 17 201940(1b)
Updated HUUCIMC info in theHyperFlex Edge and FirmwareCompatibility Matrix for 4xDeployments
September 16 201940(1b)
Release Notes for Cisco HX Data Platform Release 402
Revision History
DescriptionDateRelease
Updated HUUCIMC info in theHyperFlex Edge and FirmwareCompatibility Matrix for 3x and4x Deployments
September 10 201940(1b)
Updated HUUCIMCrecommended firmware versionsfor HyperFlex Releases 40(1b)35(2e) and 35(2d)
August 28 201940(1b)
Updated Recommended FIServerFirmware versions for HyperFlexReleases 35(2e) and 35(2d)
August 23 201940(1b)
Added Cisco IMC version supportinfo in the HyperFlex Edge andFirmware CompatibilityMatrix for4x Deployments
August 21 201940(1b)
Created release notes for Cisco HXData Platform Software Release40(1b)
August 19 201940(1b)
Added bullet describing the CiscoHyperFlex SystemsUpgradeGuidefor Unsupported Cisco HXReleases in the UpgradeGuidelines section
August 8 201940(1a)
Added important note indicatingHyperFlex does not support UCSserver firmware 40(4a) 40(4b)and 40(4c)
August 5 201940(1a)
Updated component info forHX220c M5HXAF220c M5Cluster to VIC 1457 in HyperFlexEdge and Firmware CompatibilityMatrix for 4x Deploymentssection
July 25 201940(1a)
bull AddedRelease 35(2e) supportfor ESXi 67 U2 and updatednotes for 35(2d) 35(2c) and35(2b) support for ESXi 67U2 in Supported VMwarevSphere Versions andEditions
bull Added Witness Node Versionfor Release 35(2e)
July 22 201940(1a)
Release Notes for Cisco HX Data Platform Release 403
Revision History
DescriptionDateRelease
Added CSCvq39523 to the list ofOpen Caveats for Release 40(1a)
July 5 201940(1a)
bull Updated Important Note forSED-based HyperFlexsystems in SupportedVersions and SystemRequirements section
bull Updated Release 35(2b)support for ESXi 67 U2 inSupported VMware vSphereVersions and Editions
July 1 201940(1a)
bull Added CSCvp64140 andCSCvp98910 to the list ofOpen Caveats for Release40(1a)
bull Updated HyperFlex Edge andFirmware CompatibilityMatrix tables
June 20 201940(1a)
Updated BrowserRecommendations
June 17 201940(1a)
Added information indicating40(1a) features are supported onthe Intersight Virtual Appliance andon Intersightcom
May 31 201940(1a)
bull Updated M4M5Recommended FIServerFirmware for 35(2b) and40(1a)
bull Added bullet describingrecommended use of theHypercheck Health CheckUtility in the UpgradeGuidelines section
May 21 201940(1a)
bull Added New Featuredescription for DISA STIGCompliance
bull Updated Storage ClusterSpecifications for Hyper-V
bull Added CSCvp66277 to list ofOpen Caveats
May 14 201940(1a)
Release Notes for Cisco HX Data Platform Release 404
Revision History
DescriptionDateRelease
bull Updated VMware vCenterVersions for 40(1a)
bull Updated SupportedMicrosoftSoftware versions
May 8 201940(1a)
bull Added CSCvo36198 andCSCvk38003 to the list ofResolved Caveats
bull Added CSCvp21417 to the listof Open Caveats
May 3 201940(1a)
Created release notes for Cisco HXData Platform Software Release40(1a)
April 29 201940(1a)
New FeaturesCisco HX Data Platform Release 40 provides the following features These features including the InvisibleCloud Witness for HyperFlex Edge clusters are supported both on the Intersight Virtual Appliance and onIntersightcom
New Features in Release 40(2b)
bull 76TB SSD data drivemdashSupport on HX Edge configurations
bull All NVMe and All Flash limits increasemdashLimits increased on All NVMe (1TB 4TB and 8TB) andAll Flash (76TB SSD)
bull Cluster Scale Limits increasemdashSupport for maximum scale limit increase on a cluster See Cisco HXData Platform Storage Cluster Specifications on page 16
bull HW Offload optionmdashSupport for Hardware Offload option with Stretched cluster configurations
bull Cisco Overlay Transport Virtualization for Stretched ClustermdashSupport for OTV as an overlay forStretched Cluster
New Features in Release 40(2a)
bull Cisco UCS Manager 41(1c)mdashSupport for UCSM hardware and software enhancements and featuresfrom Cisco UCSM 41(1c) release
bull Boost ModemdashThis release introduces Boost Mode for the following configurations All NVMe AllFlash C240 All Flash C220 and Hypervisor ESX Boost Mode allows the Cisco HyperFlex cluster todeliver higher IOPs by increasing the storage controller VMCPU resources by 4 vCPU For configurationinformation see the Cisco HyperFlex Data Platform Administration Guide
bull Cisco HyperFlex HTML plug-in for VMWare vCentermdashEnables virtualization administrator tomanage and monitor the Cisco HyperFlex physical infrastructure by cross launching HyperFlex Connectfrom the vSphere Client UI and perform management actions in the HyperFlex Connect UI
Release Notes for Cisco HX Data Platform Release 405
New Features
bull 25GE networking for HX EdgemdashSupport for 25GE networking for HX Edge
bull All NVMe with Stretched ClustermdashSupport for All NVMe with Stretched Cluster (ESX only)
bull Cluster Upgrade Eligibility TestmdashThis release adds the capability to perform a pre-upgrade test whichchecks for cluster readiness before upgrading Example checks include validating cluster state rebalancestatus controllerVM Free Space ESXi version and much more The Elibility test is intended to helpavoid unexpected problems that may arise during the upgrade process It is highly recommended to runthe test before performing the Hyperflex upgrade
bull Registering Smart Software LicensingmdashThis release adds support for software that allows easy trackingof the status of license and software usage trends and simplifies the three core licensing functionsPurchasing Management and Reporting
bull Dynamic self-signed certificate generation enhancementsmdashThis release adds support for Self-signedSSL certificates on the Controller VMs which were static in prior releases The static certificates arereplaced with dynamically generated self-signed certificates upon upgrading to HXDP 40(2a) so thatthe certificates are unique per cluster The new clusters installed with HXDP 40(2a) have dynamicallygenerated self-signed certificates
bull Test Upgrade EligibilitymdashThis release adds support for testing your cluster readiness and infrastructurecompatibility for an upgrade For more information see the Test Upgrade Eligibility sections in theCiscoHyperFlex SystemsUpgrade Guide for VMware ESXi Release 40 or the CiscoHyperFlex UpgradeGuide for Microsoft Hyper-V Release 40
Disaster Recovery
bull Recovery Settings ConfigurationmdashThis release supports configuration of recovery settings to defineglobal recovery parameters and mapping for resources across recovery sites These parameters are usedduring recovery test recovery and migrate operations
bull HyperFlex DR Powershell RunbookmdashPowerShell runbook functionality is extended to support therecovery configuration settings in the runbooks for all recovery scenarios New-HXrunbook cmdlet cannow be used to generate runbook for a single or multiple protection groups In addition two new cmdletsWait-HXTask and Get-HXTaskStatus are introduced
bull Protected Virtual Machine ScalabilitymdashThis release adds support for 1500 VMs across both clustersand 750 VMs per cluster in a bi-direction or any split between the two clusters without exceeding thelimit of 1500 VMs For more information see Cisco HyperFlex Data Platform Administration Guide
bull System Management REST API enhancementsmdashPause data replication actions briefly using RESTAPI to explicitly inform users on the current status of replication actions
Cisco HyperFlex with Data Platform for Hyper-V
bull Cluster-Aware Updating (CAU)mdashThis is an automated feature that allows you to perform updates onwindows servers in a failover cluster with little or no loss in availability during the upgrade process
Support for new drivesmdashNew drives are qualified for the 40(2a) release The new drives include newcapacity points and new cache drive options Several of the new drives are alternate drives to already qualifiedexisting drives in function which are qualified in 40(2a) These drives are functionally compatible with theexisting drives and are available as alternates in case of lack of availability of existing drives For expansionof existing clusters or general information about interoperability of different drives see Cisco HyperFlexDrive Compatibility
Release Notes for Cisco HX Data Platform Release 406
New Features
NVMe Caching SSDs slot information is unavailable from HX-Connect for all AF server PIDs except forthe All-NVMe server PIDs Please refer to UCSM management console for NVMe SSD slot information
Note
Applicable PlatformsDrive PIDDrive Function
All existing HX M5 serversHX-M2-960GBAlternate boot drive
All existing HX M5 servers except AllNVMe
HX-SD480G6I1X-EVAlternate system (or housekeeping) drive
All existing HX M5 servers except AllNVMe
HX-SD480GM1X-EVAlternate system (or housekeeping) drive
The following HX M5 serversHXAF220C-M5SX HXAF240C-M5SXHXAF-E-220M5SX
HX-SD800G123X-EP800G 12G SAS Cache drive option forAll Flash
All NVMe HXAF220C-M5SNHX-NVME2H-I1000All NVMe 1TB Capacity drive
All NVMe HXAF220C-M5SNHX-NVME2H-I4000All NVMe 4TB Capacity drive
All NVMe HXAF220C-M5SNHX-NVMEHW-I8000New high density All NVMe 8TBCapacity drive
All Flash Configuration ndash namelyHXAF220C-M5SX HXAF240C-M5SXHXAF-E-220M5SX
ESX support only
HX-SD76T61X-EVNew high density All Flash 76TBCapacity drive
Maximum number of drives per node is12 drives on HX AF C240
HX240C-M5LHX-HD8T7K4KANAlternate drive for 8TB LFF capacity
HXAF220C-M5SX HXAF240C-M5SXHX-SD38T2HTNK9New 38TB FIPS compliant SED SSDdata drives
HXAF220C-M5SX HXAF240C-M5SXHX-SD960G2HTNK9New 960GFIPS compliant SEDSSD datadrives
New Features in Release 40(1b)
bull Support for Second Generation Intelreg Xeonreg Scalable Processor RefreshmdashThis release includessupport for the Second Generation Intelreg Xeonreg Scalable processor refresh (formerly Cascade Lake)
New Features in Release 40(1a)
The following new features are in Release 40(1a)
bull Ultra-Light HyperFlex Edge ClustersmdashThis release introduces support for two-node HyperFlex Edgeclusters enabling HyperFlex to run in environments requiring a small footprint Cisco Intersight providescomprehensive lifecycle management and includes remote cloud-based installation centralized upgradesand invisible witnessing Both 1GE and 10GE networking topology options are available
Release Notes for Cisco HX Data Platform Release 407
New Features
bull Scaled-Up HyperFlex Edge ClustersmdashThis release adds support for four node HyperFlex Edge clustersenabling a full range of size options for remote and branch offices Size the branch office environmentsto suit current needs with a two three or four node HyperFlex Edge cluster Cisco Intersight providesfull-lifecycle management and 1GE and 10GE networking options are available
bull Cisco Intersight Invisible Cloud WitnessmdashFor two node clusters this feature eliminates the need forwitness VMs the infrastructure to run those VMs and the management overhead to deploy scale andpatch witnessing software The Invisible Cloud Witness is responsible for maintaining cluster HA in theevent of failure scenarios This feature is included at no additional cost and is automatically deployedand managed by Cisco Intersight
bull Cloud-delivered HyperFlex Edge UpgradesmdashPowered by Cisco Intersight this feature adds supportfor multi-site orchestrated remote upgrades of the HyperFlex Data Platform This feature will be enabledwith the next 40 patch release and will allowHyperFlex Edge clusters deployed via Intersight to performorchestrated upgrades across one or many sites in parallel
bull All-NVMe HyperFlexmdashStarting with this release a new high-end performance node powered by allNVMe drives is available for HyperFlex clusters Co-engineered with Intel to support Intel VMD forhot-plug and surprise removal this offering represents an industry first an enterprise-ready and fullyvalidated all-NVMe HCI appliance The all-NVMe offering is available in the 220 form factor (1RU)and is powered by Intel Optane cache drives for maximum performance and highest endurance
bull VMware Site Recovery Manager (SRM) IntegrationmdashThis release brings support for a Cisco developedStorage Replication Adapter (SRA) for SRM The SRA provides the ability to leverage HyperFlex nativeasync replication with the powerful orchestration and runbook capabilities of SRM The SRA includesthe ability to perform test recoveries planned migrations and full disaster recovery
HX SRA is certified by VMware and is available for download from VMWareSRM site
Note
bull HyperFlex DR Powershell RunbooksmdashNew Powershell cmdlets are included for automated runbookgeneration when using HyperFlex native disaster recovery The New-HXRunbook cmdlet supports thefollowing workflows Test Recovery Planned Migration and Disaster Recovery These runbooks canbe used to orchestrate DR workflows without the requirement for any third-party software
bull Windows Server 2019 with Hyper-VmdashSupport has been added in this release for the Windows Server2019 operating system for Hyper-V based HyperFlex deployments
bull Kubernetes CSI PluginmdashThis release adds support for the HyperFlex CSI (HX-CSI) plugin based onthe Kubernetes Container Storage Interface (CSI) specification Customers can now use the HX-CSIplugin to provision andmanage persistent volumes in Kubernetes version 113 and later Note Kubernetes113 support for Cisco Container Platform amp Openshift Container Platform is forthcoming in theirrespective future releases
bull C480 ML Compute only nodemdashThis release introduces support for C480 ML as a new compute-onlynode for Deep LearningMachine Learning Workloads Data scientists can now use the power of up toeight NVidia SXM2 V100 GPUs to accelerate deep learning workloads VMs running deep-learningworkloads will need to use PCIe pass-through for access to GPUs
bull Higher Capacity DrivesmdashAnew 24TB 10k rpm SAS HDD option for SFF Hybrid HyperFlex clustersand a 12TB 72K rpmSASHDDoption for LFFHybrid are now available Both HyperFlex andHyperFlexEdge support the 24TB capacity point for maximum density in this form factor Note that HyperFlex
Release Notes for Cisco HX Data Platform Release 408
New Features
Edge does not support LFF-drives HyperFlex HyperV version does not yet support the new 12TB driveoption See the HyperFlex spec sheets for a full list of configurable options
bull New Cache and increased scale for Hyper-V mdashNVMe amp Optane SSDs are now supported as cachedrives for Hyper-V deployments Furthermore scale limits have been increased to 16+16(Converged+Compute-only) for both SFF (AF and Hybrid) amp LFF (Hybrid) clusters
bull Centralized Audit Log ExportmdashThis release adds support for audit logging via a remote syslog serverThis capability enables customers to retain audit logs from all HyperFlex nodes in a centralized remotesyslog server to meet retention and compliance requirements
bull DISA STIG CompliancemdashThis release adds new HX REST APIs for setting removing and checkingstatus of DISA STIGs for Controller VMs ESXi hosts and vCenter These APIs enable customers tomeet DISA security requirements by centrally and securely applying STIGs detecting and correctingfor drifts in any STIG settings
Supported Versions and System RequirementsCisco HX Data Platform requires specific software and hardware versions and networking settings forsuccessful installation
For a complete list of requirements see
bull Cisco HyperFlex Systems Installation Guide for VMware ESXi or
bull Cisco HyperFlex Systems Installation Guide for Microsoft Hyper-V
Hardware and Software Interoperability
For a complete list of hardware and software inter-dependencies refer to respective Cisco UCS Managerrelease version of Hardware and Software Interoperability for Cisco HyperFlex HX-Series
HyperFlex Software Versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinan HX Storage Cluster are compatible
bull HyperFlex does not support UCS Manager and UCS Server Firmware versions 40(4a) 40(4b)and 40(4c)
Do not upgrade to these versions of firmware
Do not upgrade to these versions of UCS Manager
Important
bull Verify that the preconfigured HX servers have the same version of Cisco UCS server firmware installedIf the Cisco UCS Fabric Interconnects (FI) firmware versions are different see the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40 for steps to align the firmware versions
bull M4 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M4 or HX220c M4) deploymentsverify that Cisco UCS Manager 31(3k) 32(3i) or 40(2b) or higher is installed For moreinformation see Recommended Cisco HyperFlex HX Data Platform Software Releases
Release Notes for Cisco HX Data Platform Release 409
Supported Versions and System Requirements
bull M5 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M5 or HX220c M5) deploymentsverify that the recommended UCS firmware version is installed
For SED-based HyperFlex systems ensure that the A (Infrastructure) B (Bladeserver) and C (Rack server) bundles are at Cisco UCS Manager version 40(2b)or later for all SED M4M5 systems For more details see CSCvh04307 ForSED-based HyperFlex systems also ensure that all clusters are at HyperFlexRelease 35(2b) or later For more information see Field Notice (70234) andCSCvk17250
Important
bull To reinstall an HX server download supported and compatible versions of the software See theCisco HyperFlex Systems Installation Guide for VMware ESXi for the requirements and steps
bull Important For Intersight edge servers running older than 40(1a) CIMC version HUU is the suggestedmechanism to update the firmware
Table 1 HyperFlex Software Versions for M4M5 Servers
M5 Recommended FIServer Firmware
(be sure to review important notes above)
M4 Recommended FIServer Firmware
(be sure to review important notes above)
HyperFlexRelease
40(4h)40(4h)40(2b)
40(4h)40(4h)40(2a)
40(4h)40(4h)40(1b)
40(4e)40(4e)40(1a)
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Cisco HX Data Platform Release 4x based Deployments
Confirm the component firmware on the server meets the minimum versions listed in the following tables
HyperFlex Edge does not support Cisco IMC versions 40(4a) 40(4b) 40(4c) 40(4d) and 40(4e)Important
Table 2 HX220c M4 HXAF220c M4 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
402d0BIOS
40(2h)Cisco Integrated Management Controller(CIMC)
Release Notes for Cisco HX Data Platform Release 4010
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
13000012SAS HBA
43(2b)VIC 1227
0103 (Toshiba)
CS07 (Intel 480)
SSD
CN03 (Seagate)
CK03 (Seagate 18)
CK03 (Seagate 24)
sF3Q (Samsung)
3F3Q (Samsung)
HDD
40(2h)
Download Software
Host Upgrade Utility (HUU) Download Link
Table 3 HX220c M5 HXAF220c M5 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
C220M5404p00224200755BIOS
40(4i)Cisco Integrated Management Controller(CIMC)
09000008SAS HBA
43(3c)VIC 1387
50(3e)VIC 1457
G201CS01 (Intel)
0104 (Toshiba)
0107 (Toshiba)
SSD
2F3Q (Samsung)
3F3Q (Samsung)
NOA4 (Seagate)
CK03 (Seagate 18tb)
CK03 (Seagate 24)
HDD
Release Notes for Cisco HX Data Platform Release 4011
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
40(4k)
Download Software
Host Upgrade Utility (HUU) Download Link
HyperFlex Licensing
As of version 26(1a) HyperFlex supports VMware PAC licensing Existing VMware embedded licenses willcontinue to be supported
As of version 25(1a) HyperFlex uses a smart licensing mechanism to apply your licenses See the CiscoHyperFlex Systems Installation Guide for VMware ESXi for details and steps
VMware vSphere Licensing Requirements
How you purchase your vSphere license determines how your license applies to your HyperFlex system
bull If you purchased your vSphere license with HyperFlex
Each HyperFlex server either has the Enterprise or Enterprise Plus edition preinstalled at the factory
bull HX Nodes have OEM licenses preinstalled If you delete or overwrite thecontent of the boot drives after receiving the HX servers you also delete thefactory-installed licenses
bull OEM license keys is a newVMware vCenter 60 U1b feature Earlier versionsdo not support OEM licenses
bull All factory-installed HX nodes share the same OEM license key WithvSphere OEM keys the Usage count can exceed the Capacity value
bull When you add an HX host to vCenter through the Add Host wizard in theAssign license section select the OEM license
We obfuscate the actual vSphere OEM license key for example0N085-XXXXX-XXXXX-XXXXX-10LHH
bull Standard Essentials Plus and ROBO editions are not available preinstalledon HX servers
Note
bull If you did NOT purchase your vSphere license with HyperFlex
The HX nodes have a vSphere Foundation license preinstalled After initial setup you can apply thelicense to a supported version of vSphere
bull If you purchased a vSphere PAC license
Follow the instructions in your PAC license letter from VMware to add the license to your MY VMwareaccount then follow the instructions to add your HX host to vCenter and assign the PAC license
Release Notes for Cisco HX Data Platform Release 4012
HyperFlex Licensing
HX Data Platform Software Versions for HyperFlex Witness Node
Witness Node VersionHyperFlex Release
10840(2b)
10840(2a)
10440(1b)
10440(1a)
Software Requirements for VMware ESXi
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and VMware vSphere VMware vCenter and VMware ESXi
bull Verify that all HX servers have a compatible version of vSphere preinstalled
bull Verify that the vCenter version is the same or later than the ESXi version
bull Verify that the vCenter and ESXi versions are compatible by consulting the VMware ProductInteroperability Matrix Newer vCenter versions may be used with older ESXi versions so long as bothESXi and vCenter are supported in the table below
bull Verify that you have a vCenter administrator account with root-level privileges and the associatedpassword
For VIC1457 there is no support for ESXi 60Note
The below table applies for all of the following VMware vSphere Editions Enterprise Enterprise PlusStandard Essentials Plus ROBO
VMware vCenter VersionsVMware ESXi VersionsHyperFlex Version
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2b)1
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2a)2
60 U3 65 U3 67 U260 U3 65 U3 67 U2340(1b)
60 U3 65 U2 67 U260 U3 65 U2 67 U2440(1a)
1 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
2 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
3 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
4 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4013
HX Data Platform Software Versions for HyperFlex Witness Node
Software Requirements for Microsoft Hyper-V
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and Microsoft Hyper-V (Hyper-V) components
HyperFlex Software versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinthe HX Storage Cluster are compatible
bull Cisco HyperFlex M5 Converged nodesmdash For Hybrid (Cisco HyperFlex HX240c M5 HX220c M5)and All Flash (Cisco HyperFlex HXAF240cM5 HXAF220cM5) verify that Cisco UCSManager 40(2b)is installed HX 40 (1a) does not support Hyper-V on the All NVMe (HXAF220C-M5SN) nodes Fordetailed information on installation requirements and steps see theCisco HyperFlex Systems InstallationGuide on Microsoft Hyper-V
Table 4 Supported HyperFlex Software versions for M5 Servers on Hyper-V
M5 Recommended Server FirmwareHyperFlex Release
40(4h)40(2b)
40(4g)40(2a)
40(4e)40(1b)
40(4e)40(1a)
Release Notes for Cisco HX Data Platform Release 4014
Software Requirements for Microsoft Hyper-V
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DescriptionDateRelease
Updated M4 and M5Recommended FIServer Firmwareto UCS 40(4h) for HX 40(2a)
March 24 202040(2a)
Created release notes for Cisco HXData Platform Software Release40(2a)
February 11 202040(2a)
Updated release notes for deferredCisco HyperFlex Release HX35(2c)
January 15 202035(2c)
Updated M4 and M5Recommended FIServer Firmwareto UCS 40(4e) for HX 40(1b)40(1a) 35(2f) 35(2e) and35(2d)
December 23 201940(1b)
Added CSCvs28167 to the list ofResolved Caveats for HX 40(1b)and HX 40(1a)
December 13 201940(1b)
Added CSCvs02466 to the list ofOpen Caveats
November 25 201940(1b)
Updated info in the HyperFlexEdge and Firmware CompatibilityMatrix for 3x Deployments
Updated info in the Storage ClusterSpecifications
November 7 201940(1b)
Added CSCvj95606 andCSCvq24176 to the list of SecurityFixes
October 25 201940(1b)
Updated Recommended FIServerFirmware versions
October 8 201940(1b)
Updated HUUCIMC info in theHyperFlex Edge and FirmwareCompatibility Matrix for 4xDeployments
September 30 201940(1b)
AddedCSCvq41985 to new sectionfor Related Caveats
September 17 201940(1b)
Updated HUUCIMC info in theHyperFlex Edge and FirmwareCompatibility Matrix for 4xDeployments
September 16 201940(1b)
Release Notes for Cisco HX Data Platform Release 402
Revision History
DescriptionDateRelease
Updated HUUCIMC info in theHyperFlex Edge and FirmwareCompatibility Matrix for 3x and4x Deployments
September 10 201940(1b)
Updated HUUCIMCrecommended firmware versionsfor HyperFlex Releases 40(1b)35(2e) and 35(2d)
August 28 201940(1b)
Updated Recommended FIServerFirmware versions for HyperFlexReleases 35(2e) and 35(2d)
August 23 201940(1b)
Added Cisco IMC version supportinfo in the HyperFlex Edge andFirmware CompatibilityMatrix for4x Deployments
August 21 201940(1b)
Created release notes for Cisco HXData Platform Software Release40(1b)
August 19 201940(1b)
Added bullet describing the CiscoHyperFlex SystemsUpgradeGuidefor Unsupported Cisco HXReleases in the UpgradeGuidelines section
August 8 201940(1a)
Added important note indicatingHyperFlex does not support UCSserver firmware 40(4a) 40(4b)and 40(4c)
August 5 201940(1a)
Updated component info forHX220c M5HXAF220c M5Cluster to VIC 1457 in HyperFlexEdge and Firmware CompatibilityMatrix for 4x Deploymentssection
July 25 201940(1a)
bull AddedRelease 35(2e) supportfor ESXi 67 U2 and updatednotes for 35(2d) 35(2c) and35(2b) support for ESXi 67U2 in Supported VMwarevSphere Versions andEditions
bull Added Witness Node Versionfor Release 35(2e)
July 22 201940(1a)
Release Notes for Cisco HX Data Platform Release 403
Revision History
DescriptionDateRelease
Added CSCvq39523 to the list ofOpen Caveats for Release 40(1a)
July 5 201940(1a)
bull Updated Important Note forSED-based HyperFlexsystems in SupportedVersions and SystemRequirements section
bull Updated Release 35(2b)support for ESXi 67 U2 inSupported VMware vSphereVersions and Editions
July 1 201940(1a)
bull Added CSCvp64140 andCSCvp98910 to the list ofOpen Caveats for Release40(1a)
bull Updated HyperFlex Edge andFirmware CompatibilityMatrix tables
June 20 201940(1a)
Updated BrowserRecommendations
June 17 201940(1a)
Added information indicating40(1a) features are supported onthe Intersight Virtual Appliance andon Intersightcom
May 31 201940(1a)
bull Updated M4M5Recommended FIServerFirmware for 35(2b) and40(1a)
bull Added bullet describingrecommended use of theHypercheck Health CheckUtility in the UpgradeGuidelines section
May 21 201940(1a)
bull Added New Featuredescription for DISA STIGCompliance
bull Updated Storage ClusterSpecifications for Hyper-V
bull Added CSCvp66277 to list ofOpen Caveats
May 14 201940(1a)
Release Notes for Cisco HX Data Platform Release 404
Revision History
DescriptionDateRelease
bull Updated VMware vCenterVersions for 40(1a)
bull Updated SupportedMicrosoftSoftware versions
May 8 201940(1a)
bull Added CSCvo36198 andCSCvk38003 to the list ofResolved Caveats
bull Added CSCvp21417 to the listof Open Caveats
May 3 201940(1a)
Created release notes for Cisco HXData Platform Software Release40(1a)
April 29 201940(1a)
New FeaturesCisco HX Data Platform Release 40 provides the following features These features including the InvisibleCloud Witness for HyperFlex Edge clusters are supported both on the Intersight Virtual Appliance and onIntersightcom
New Features in Release 40(2b)
bull 76TB SSD data drivemdashSupport on HX Edge configurations
bull All NVMe and All Flash limits increasemdashLimits increased on All NVMe (1TB 4TB and 8TB) andAll Flash (76TB SSD)
bull Cluster Scale Limits increasemdashSupport for maximum scale limit increase on a cluster See Cisco HXData Platform Storage Cluster Specifications on page 16
bull HW Offload optionmdashSupport for Hardware Offload option with Stretched cluster configurations
bull Cisco Overlay Transport Virtualization for Stretched ClustermdashSupport for OTV as an overlay forStretched Cluster
New Features in Release 40(2a)
bull Cisco UCS Manager 41(1c)mdashSupport for UCSM hardware and software enhancements and featuresfrom Cisco UCSM 41(1c) release
bull Boost ModemdashThis release introduces Boost Mode for the following configurations All NVMe AllFlash C240 All Flash C220 and Hypervisor ESX Boost Mode allows the Cisco HyperFlex cluster todeliver higher IOPs by increasing the storage controller VMCPU resources by 4 vCPU For configurationinformation see the Cisco HyperFlex Data Platform Administration Guide
bull Cisco HyperFlex HTML plug-in for VMWare vCentermdashEnables virtualization administrator tomanage and monitor the Cisco HyperFlex physical infrastructure by cross launching HyperFlex Connectfrom the vSphere Client UI and perform management actions in the HyperFlex Connect UI
Release Notes for Cisco HX Data Platform Release 405
New Features
bull 25GE networking for HX EdgemdashSupport for 25GE networking for HX Edge
bull All NVMe with Stretched ClustermdashSupport for All NVMe with Stretched Cluster (ESX only)
bull Cluster Upgrade Eligibility TestmdashThis release adds the capability to perform a pre-upgrade test whichchecks for cluster readiness before upgrading Example checks include validating cluster state rebalancestatus controllerVM Free Space ESXi version and much more The Elibility test is intended to helpavoid unexpected problems that may arise during the upgrade process It is highly recommended to runthe test before performing the Hyperflex upgrade
bull Registering Smart Software LicensingmdashThis release adds support for software that allows easy trackingof the status of license and software usage trends and simplifies the three core licensing functionsPurchasing Management and Reporting
bull Dynamic self-signed certificate generation enhancementsmdashThis release adds support for Self-signedSSL certificates on the Controller VMs which were static in prior releases The static certificates arereplaced with dynamically generated self-signed certificates upon upgrading to HXDP 40(2a) so thatthe certificates are unique per cluster The new clusters installed with HXDP 40(2a) have dynamicallygenerated self-signed certificates
bull Test Upgrade EligibilitymdashThis release adds support for testing your cluster readiness and infrastructurecompatibility for an upgrade For more information see the Test Upgrade Eligibility sections in theCiscoHyperFlex SystemsUpgrade Guide for VMware ESXi Release 40 or the CiscoHyperFlex UpgradeGuide for Microsoft Hyper-V Release 40
Disaster Recovery
bull Recovery Settings ConfigurationmdashThis release supports configuration of recovery settings to defineglobal recovery parameters and mapping for resources across recovery sites These parameters are usedduring recovery test recovery and migrate operations
bull HyperFlex DR Powershell RunbookmdashPowerShell runbook functionality is extended to support therecovery configuration settings in the runbooks for all recovery scenarios New-HXrunbook cmdlet cannow be used to generate runbook for a single or multiple protection groups In addition two new cmdletsWait-HXTask and Get-HXTaskStatus are introduced
bull Protected Virtual Machine ScalabilitymdashThis release adds support for 1500 VMs across both clustersand 750 VMs per cluster in a bi-direction or any split between the two clusters without exceeding thelimit of 1500 VMs For more information see Cisco HyperFlex Data Platform Administration Guide
bull System Management REST API enhancementsmdashPause data replication actions briefly using RESTAPI to explicitly inform users on the current status of replication actions
Cisco HyperFlex with Data Platform for Hyper-V
bull Cluster-Aware Updating (CAU)mdashThis is an automated feature that allows you to perform updates onwindows servers in a failover cluster with little or no loss in availability during the upgrade process
Support for new drivesmdashNew drives are qualified for the 40(2a) release The new drives include newcapacity points and new cache drive options Several of the new drives are alternate drives to already qualifiedexisting drives in function which are qualified in 40(2a) These drives are functionally compatible with theexisting drives and are available as alternates in case of lack of availability of existing drives For expansionof existing clusters or general information about interoperability of different drives see Cisco HyperFlexDrive Compatibility
Release Notes for Cisco HX Data Platform Release 406
New Features
NVMe Caching SSDs slot information is unavailable from HX-Connect for all AF server PIDs except forthe All-NVMe server PIDs Please refer to UCSM management console for NVMe SSD slot information
Note
Applicable PlatformsDrive PIDDrive Function
All existing HX M5 serversHX-M2-960GBAlternate boot drive
All existing HX M5 servers except AllNVMe
HX-SD480G6I1X-EVAlternate system (or housekeeping) drive
All existing HX M5 servers except AllNVMe
HX-SD480GM1X-EVAlternate system (or housekeeping) drive
The following HX M5 serversHXAF220C-M5SX HXAF240C-M5SXHXAF-E-220M5SX
HX-SD800G123X-EP800G 12G SAS Cache drive option forAll Flash
All NVMe HXAF220C-M5SNHX-NVME2H-I1000All NVMe 1TB Capacity drive
All NVMe HXAF220C-M5SNHX-NVME2H-I4000All NVMe 4TB Capacity drive
All NVMe HXAF220C-M5SNHX-NVMEHW-I8000New high density All NVMe 8TBCapacity drive
All Flash Configuration ndash namelyHXAF220C-M5SX HXAF240C-M5SXHXAF-E-220M5SX
ESX support only
HX-SD76T61X-EVNew high density All Flash 76TBCapacity drive
Maximum number of drives per node is12 drives on HX AF C240
HX240C-M5LHX-HD8T7K4KANAlternate drive for 8TB LFF capacity
HXAF220C-M5SX HXAF240C-M5SXHX-SD38T2HTNK9New 38TB FIPS compliant SED SSDdata drives
HXAF220C-M5SX HXAF240C-M5SXHX-SD960G2HTNK9New 960GFIPS compliant SEDSSD datadrives
New Features in Release 40(1b)
bull Support for Second Generation Intelreg Xeonreg Scalable Processor RefreshmdashThis release includessupport for the Second Generation Intelreg Xeonreg Scalable processor refresh (formerly Cascade Lake)
New Features in Release 40(1a)
The following new features are in Release 40(1a)
bull Ultra-Light HyperFlex Edge ClustersmdashThis release introduces support for two-node HyperFlex Edgeclusters enabling HyperFlex to run in environments requiring a small footprint Cisco Intersight providescomprehensive lifecycle management and includes remote cloud-based installation centralized upgradesand invisible witnessing Both 1GE and 10GE networking topology options are available
Release Notes for Cisco HX Data Platform Release 407
New Features
bull Scaled-Up HyperFlex Edge ClustersmdashThis release adds support for four node HyperFlex Edge clustersenabling a full range of size options for remote and branch offices Size the branch office environmentsto suit current needs with a two three or four node HyperFlex Edge cluster Cisco Intersight providesfull-lifecycle management and 1GE and 10GE networking options are available
bull Cisco Intersight Invisible Cloud WitnessmdashFor two node clusters this feature eliminates the need forwitness VMs the infrastructure to run those VMs and the management overhead to deploy scale andpatch witnessing software The Invisible Cloud Witness is responsible for maintaining cluster HA in theevent of failure scenarios This feature is included at no additional cost and is automatically deployedand managed by Cisco Intersight
bull Cloud-delivered HyperFlex Edge UpgradesmdashPowered by Cisco Intersight this feature adds supportfor multi-site orchestrated remote upgrades of the HyperFlex Data Platform This feature will be enabledwith the next 40 patch release and will allowHyperFlex Edge clusters deployed via Intersight to performorchestrated upgrades across one or many sites in parallel
bull All-NVMe HyperFlexmdashStarting with this release a new high-end performance node powered by allNVMe drives is available for HyperFlex clusters Co-engineered with Intel to support Intel VMD forhot-plug and surprise removal this offering represents an industry first an enterprise-ready and fullyvalidated all-NVMe HCI appliance The all-NVMe offering is available in the 220 form factor (1RU)and is powered by Intel Optane cache drives for maximum performance and highest endurance
bull VMware Site Recovery Manager (SRM) IntegrationmdashThis release brings support for a Cisco developedStorage Replication Adapter (SRA) for SRM The SRA provides the ability to leverage HyperFlex nativeasync replication with the powerful orchestration and runbook capabilities of SRM The SRA includesthe ability to perform test recoveries planned migrations and full disaster recovery
HX SRA is certified by VMware and is available for download from VMWareSRM site
Note
bull HyperFlex DR Powershell RunbooksmdashNew Powershell cmdlets are included for automated runbookgeneration when using HyperFlex native disaster recovery The New-HXRunbook cmdlet supports thefollowing workflows Test Recovery Planned Migration and Disaster Recovery These runbooks canbe used to orchestrate DR workflows without the requirement for any third-party software
bull Windows Server 2019 with Hyper-VmdashSupport has been added in this release for the Windows Server2019 operating system for Hyper-V based HyperFlex deployments
bull Kubernetes CSI PluginmdashThis release adds support for the HyperFlex CSI (HX-CSI) plugin based onthe Kubernetes Container Storage Interface (CSI) specification Customers can now use the HX-CSIplugin to provision andmanage persistent volumes in Kubernetes version 113 and later Note Kubernetes113 support for Cisco Container Platform amp Openshift Container Platform is forthcoming in theirrespective future releases
bull C480 ML Compute only nodemdashThis release introduces support for C480 ML as a new compute-onlynode for Deep LearningMachine Learning Workloads Data scientists can now use the power of up toeight NVidia SXM2 V100 GPUs to accelerate deep learning workloads VMs running deep-learningworkloads will need to use PCIe pass-through for access to GPUs
bull Higher Capacity DrivesmdashAnew 24TB 10k rpm SAS HDD option for SFF Hybrid HyperFlex clustersand a 12TB 72K rpmSASHDDoption for LFFHybrid are now available Both HyperFlex andHyperFlexEdge support the 24TB capacity point for maximum density in this form factor Note that HyperFlex
Release Notes for Cisco HX Data Platform Release 408
New Features
Edge does not support LFF-drives HyperFlex HyperV version does not yet support the new 12TB driveoption See the HyperFlex spec sheets for a full list of configurable options
bull New Cache and increased scale for Hyper-V mdashNVMe amp Optane SSDs are now supported as cachedrives for Hyper-V deployments Furthermore scale limits have been increased to 16+16(Converged+Compute-only) for both SFF (AF and Hybrid) amp LFF (Hybrid) clusters
bull Centralized Audit Log ExportmdashThis release adds support for audit logging via a remote syslog serverThis capability enables customers to retain audit logs from all HyperFlex nodes in a centralized remotesyslog server to meet retention and compliance requirements
bull DISA STIG CompliancemdashThis release adds new HX REST APIs for setting removing and checkingstatus of DISA STIGs for Controller VMs ESXi hosts and vCenter These APIs enable customers tomeet DISA security requirements by centrally and securely applying STIGs detecting and correctingfor drifts in any STIG settings
Supported Versions and System RequirementsCisco HX Data Platform requires specific software and hardware versions and networking settings forsuccessful installation
For a complete list of requirements see
bull Cisco HyperFlex Systems Installation Guide for VMware ESXi or
bull Cisco HyperFlex Systems Installation Guide for Microsoft Hyper-V
Hardware and Software Interoperability
For a complete list of hardware and software inter-dependencies refer to respective Cisco UCS Managerrelease version of Hardware and Software Interoperability for Cisco HyperFlex HX-Series
HyperFlex Software Versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinan HX Storage Cluster are compatible
bull HyperFlex does not support UCS Manager and UCS Server Firmware versions 40(4a) 40(4b)and 40(4c)
Do not upgrade to these versions of firmware
Do not upgrade to these versions of UCS Manager
Important
bull Verify that the preconfigured HX servers have the same version of Cisco UCS server firmware installedIf the Cisco UCS Fabric Interconnects (FI) firmware versions are different see the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40 for steps to align the firmware versions
bull M4 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M4 or HX220c M4) deploymentsverify that Cisco UCS Manager 31(3k) 32(3i) or 40(2b) or higher is installed For moreinformation see Recommended Cisco HyperFlex HX Data Platform Software Releases
Release Notes for Cisco HX Data Platform Release 409
Supported Versions and System Requirements
bull M5 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M5 or HX220c M5) deploymentsverify that the recommended UCS firmware version is installed
For SED-based HyperFlex systems ensure that the A (Infrastructure) B (Bladeserver) and C (Rack server) bundles are at Cisco UCS Manager version 40(2b)or later for all SED M4M5 systems For more details see CSCvh04307 ForSED-based HyperFlex systems also ensure that all clusters are at HyperFlexRelease 35(2b) or later For more information see Field Notice (70234) andCSCvk17250
Important
bull To reinstall an HX server download supported and compatible versions of the software See theCisco HyperFlex Systems Installation Guide for VMware ESXi for the requirements and steps
bull Important For Intersight edge servers running older than 40(1a) CIMC version HUU is the suggestedmechanism to update the firmware
Table 1 HyperFlex Software Versions for M4M5 Servers
M5 Recommended FIServer Firmware
(be sure to review important notes above)
M4 Recommended FIServer Firmware
(be sure to review important notes above)
HyperFlexRelease
40(4h)40(4h)40(2b)
40(4h)40(4h)40(2a)
40(4h)40(4h)40(1b)
40(4e)40(4e)40(1a)
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Cisco HX Data Platform Release 4x based Deployments
Confirm the component firmware on the server meets the minimum versions listed in the following tables
HyperFlex Edge does not support Cisco IMC versions 40(4a) 40(4b) 40(4c) 40(4d) and 40(4e)Important
Table 2 HX220c M4 HXAF220c M4 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
402d0BIOS
40(2h)Cisco Integrated Management Controller(CIMC)
Release Notes for Cisco HX Data Platform Release 4010
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
13000012SAS HBA
43(2b)VIC 1227
0103 (Toshiba)
CS07 (Intel 480)
SSD
CN03 (Seagate)
CK03 (Seagate 18)
CK03 (Seagate 24)
sF3Q (Samsung)
3F3Q (Samsung)
HDD
40(2h)
Download Software
Host Upgrade Utility (HUU) Download Link
Table 3 HX220c M5 HXAF220c M5 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
C220M5404p00224200755BIOS
40(4i)Cisco Integrated Management Controller(CIMC)
09000008SAS HBA
43(3c)VIC 1387
50(3e)VIC 1457
G201CS01 (Intel)
0104 (Toshiba)
0107 (Toshiba)
SSD
2F3Q (Samsung)
3F3Q (Samsung)
NOA4 (Seagate)
CK03 (Seagate 18tb)
CK03 (Seagate 24)
HDD
Release Notes for Cisco HX Data Platform Release 4011
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
40(4k)
Download Software
Host Upgrade Utility (HUU) Download Link
HyperFlex Licensing
As of version 26(1a) HyperFlex supports VMware PAC licensing Existing VMware embedded licenses willcontinue to be supported
As of version 25(1a) HyperFlex uses a smart licensing mechanism to apply your licenses See the CiscoHyperFlex Systems Installation Guide for VMware ESXi for details and steps
VMware vSphere Licensing Requirements
How you purchase your vSphere license determines how your license applies to your HyperFlex system
bull If you purchased your vSphere license with HyperFlex
Each HyperFlex server either has the Enterprise or Enterprise Plus edition preinstalled at the factory
bull HX Nodes have OEM licenses preinstalled If you delete or overwrite thecontent of the boot drives after receiving the HX servers you also delete thefactory-installed licenses
bull OEM license keys is a newVMware vCenter 60 U1b feature Earlier versionsdo not support OEM licenses
bull All factory-installed HX nodes share the same OEM license key WithvSphere OEM keys the Usage count can exceed the Capacity value
bull When you add an HX host to vCenter through the Add Host wizard in theAssign license section select the OEM license
We obfuscate the actual vSphere OEM license key for example0N085-XXXXX-XXXXX-XXXXX-10LHH
bull Standard Essentials Plus and ROBO editions are not available preinstalledon HX servers
Note
bull If you did NOT purchase your vSphere license with HyperFlex
The HX nodes have a vSphere Foundation license preinstalled After initial setup you can apply thelicense to a supported version of vSphere
bull If you purchased a vSphere PAC license
Follow the instructions in your PAC license letter from VMware to add the license to your MY VMwareaccount then follow the instructions to add your HX host to vCenter and assign the PAC license
Release Notes for Cisco HX Data Platform Release 4012
HyperFlex Licensing
HX Data Platform Software Versions for HyperFlex Witness Node
Witness Node VersionHyperFlex Release
10840(2b)
10840(2a)
10440(1b)
10440(1a)
Software Requirements for VMware ESXi
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and VMware vSphere VMware vCenter and VMware ESXi
bull Verify that all HX servers have a compatible version of vSphere preinstalled
bull Verify that the vCenter version is the same or later than the ESXi version
bull Verify that the vCenter and ESXi versions are compatible by consulting the VMware ProductInteroperability Matrix Newer vCenter versions may be used with older ESXi versions so long as bothESXi and vCenter are supported in the table below
bull Verify that you have a vCenter administrator account with root-level privileges and the associatedpassword
For VIC1457 there is no support for ESXi 60Note
The below table applies for all of the following VMware vSphere Editions Enterprise Enterprise PlusStandard Essentials Plus ROBO
VMware vCenter VersionsVMware ESXi VersionsHyperFlex Version
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2b)1
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2a)2
60 U3 65 U3 67 U260 U3 65 U3 67 U2340(1b)
60 U3 65 U2 67 U260 U3 65 U2 67 U2440(1a)
1 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
2 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
3 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
4 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4013
HX Data Platform Software Versions for HyperFlex Witness Node
Software Requirements for Microsoft Hyper-V
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and Microsoft Hyper-V (Hyper-V) components
HyperFlex Software versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinthe HX Storage Cluster are compatible
bull Cisco HyperFlex M5 Converged nodesmdash For Hybrid (Cisco HyperFlex HX240c M5 HX220c M5)and All Flash (Cisco HyperFlex HXAF240cM5 HXAF220cM5) verify that Cisco UCSManager 40(2b)is installed HX 40 (1a) does not support Hyper-V on the All NVMe (HXAF220C-M5SN) nodes Fordetailed information on installation requirements and steps see theCisco HyperFlex Systems InstallationGuide on Microsoft Hyper-V
Table 4 Supported HyperFlex Software versions for M5 Servers on Hyper-V
M5 Recommended Server FirmwareHyperFlex Release
40(4h)40(2b)
40(4g)40(2a)
40(4e)40(1b)
40(4e)40(1a)
Release Notes for Cisco HX Data Platform Release 4014
Software Requirements for Microsoft Hyper-V
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DescriptionDateRelease
Updated HUUCIMC info in theHyperFlex Edge and FirmwareCompatibility Matrix for 3x and4x Deployments
September 10 201940(1b)
Updated HUUCIMCrecommended firmware versionsfor HyperFlex Releases 40(1b)35(2e) and 35(2d)
August 28 201940(1b)
Updated Recommended FIServerFirmware versions for HyperFlexReleases 35(2e) and 35(2d)
August 23 201940(1b)
Added Cisco IMC version supportinfo in the HyperFlex Edge andFirmware CompatibilityMatrix for4x Deployments
August 21 201940(1b)
Created release notes for Cisco HXData Platform Software Release40(1b)
August 19 201940(1b)
Added bullet describing the CiscoHyperFlex SystemsUpgradeGuidefor Unsupported Cisco HXReleases in the UpgradeGuidelines section
August 8 201940(1a)
Added important note indicatingHyperFlex does not support UCSserver firmware 40(4a) 40(4b)and 40(4c)
August 5 201940(1a)
Updated component info forHX220c M5HXAF220c M5Cluster to VIC 1457 in HyperFlexEdge and Firmware CompatibilityMatrix for 4x Deploymentssection
July 25 201940(1a)
bull AddedRelease 35(2e) supportfor ESXi 67 U2 and updatednotes for 35(2d) 35(2c) and35(2b) support for ESXi 67U2 in Supported VMwarevSphere Versions andEditions
bull Added Witness Node Versionfor Release 35(2e)
July 22 201940(1a)
Release Notes for Cisco HX Data Platform Release 403
Revision History
DescriptionDateRelease
Added CSCvq39523 to the list ofOpen Caveats for Release 40(1a)
July 5 201940(1a)
bull Updated Important Note forSED-based HyperFlexsystems in SupportedVersions and SystemRequirements section
bull Updated Release 35(2b)support for ESXi 67 U2 inSupported VMware vSphereVersions and Editions
July 1 201940(1a)
bull Added CSCvp64140 andCSCvp98910 to the list ofOpen Caveats for Release40(1a)
bull Updated HyperFlex Edge andFirmware CompatibilityMatrix tables
June 20 201940(1a)
Updated BrowserRecommendations
June 17 201940(1a)
Added information indicating40(1a) features are supported onthe Intersight Virtual Appliance andon Intersightcom
May 31 201940(1a)
bull Updated M4M5Recommended FIServerFirmware for 35(2b) and40(1a)
bull Added bullet describingrecommended use of theHypercheck Health CheckUtility in the UpgradeGuidelines section
May 21 201940(1a)
bull Added New Featuredescription for DISA STIGCompliance
bull Updated Storage ClusterSpecifications for Hyper-V
bull Added CSCvp66277 to list ofOpen Caveats
May 14 201940(1a)
Release Notes for Cisco HX Data Platform Release 404
Revision History
DescriptionDateRelease
bull Updated VMware vCenterVersions for 40(1a)
bull Updated SupportedMicrosoftSoftware versions
May 8 201940(1a)
bull Added CSCvo36198 andCSCvk38003 to the list ofResolved Caveats
bull Added CSCvp21417 to the listof Open Caveats
May 3 201940(1a)
Created release notes for Cisco HXData Platform Software Release40(1a)
April 29 201940(1a)
New FeaturesCisco HX Data Platform Release 40 provides the following features These features including the InvisibleCloud Witness for HyperFlex Edge clusters are supported both on the Intersight Virtual Appliance and onIntersightcom
New Features in Release 40(2b)
bull 76TB SSD data drivemdashSupport on HX Edge configurations
bull All NVMe and All Flash limits increasemdashLimits increased on All NVMe (1TB 4TB and 8TB) andAll Flash (76TB SSD)
bull Cluster Scale Limits increasemdashSupport for maximum scale limit increase on a cluster See Cisco HXData Platform Storage Cluster Specifications on page 16
bull HW Offload optionmdashSupport for Hardware Offload option with Stretched cluster configurations
bull Cisco Overlay Transport Virtualization for Stretched ClustermdashSupport for OTV as an overlay forStretched Cluster
New Features in Release 40(2a)
bull Cisco UCS Manager 41(1c)mdashSupport for UCSM hardware and software enhancements and featuresfrom Cisco UCSM 41(1c) release
bull Boost ModemdashThis release introduces Boost Mode for the following configurations All NVMe AllFlash C240 All Flash C220 and Hypervisor ESX Boost Mode allows the Cisco HyperFlex cluster todeliver higher IOPs by increasing the storage controller VMCPU resources by 4 vCPU For configurationinformation see the Cisco HyperFlex Data Platform Administration Guide
bull Cisco HyperFlex HTML plug-in for VMWare vCentermdashEnables virtualization administrator tomanage and monitor the Cisco HyperFlex physical infrastructure by cross launching HyperFlex Connectfrom the vSphere Client UI and perform management actions in the HyperFlex Connect UI
Release Notes for Cisco HX Data Platform Release 405
New Features
bull 25GE networking for HX EdgemdashSupport for 25GE networking for HX Edge
bull All NVMe with Stretched ClustermdashSupport for All NVMe with Stretched Cluster (ESX only)
bull Cluster Upgrade Eligibility TestmdashThis release adds the capability to perform a pre-upgrade test whichchecks for cluster readiness before upgrading Example checks include validating cluster state rebalancestatus controllerVM Free Space ESXi version and much more The Elibility test is intended to helpavoid unexpected problems that may arise during the upgrade process It is highly recommended to runthe test before performing the Hyperflex upgrade
bull Registering Smart Software LicensingmdashThis release adds support for software that allows easy trackingof the status of license and software usage trends and simplifies the three core licensing functionsPurchasing Management and Reporting
bull Dynamic self-signed certificate generation enhancementsmdashThis release adds support for Self-signedSSL certificates on the Controller VMs which were static in prior releases The static certificates arereplaced with dynamically generated self-signed certificates upon upgrading to HXDP 40(2a) so thatthe certificates are unique per cluster The new clusters installed with HXDP 40(2a) have dynamicallygenerated self-signed certificates
bull Test Upgrade EligibilitymdashThis release adds support for testing your cluster readiness and infrastructurecompatibility for an upgrade For more information see the Test Upgrade Eligibility sections in theCiscoHyperFlex SystemsUpgrade Guide for VMware ESXi Release 40 or the CiscoHyperFlex UpgradeGuide for Microsoft Hyper-V Release 40
Disaster Recovery
bull Recovery Settings ConfigurationmdashThis release supports configuration of recovery settings to defineglobal recovery parameters and mapping for resources across recovery sites These parameters are usedduring recovery test recovery and migrate operations
bull HyperFlex DR Powershell RunbookmdashPowerShell runbook functionality is extended to support therecovery configuration settings in the runbooks for all recovery scenarios New-HXrunbook cmdlet cannow be used to generate runbook for a single or multiple protection groups In addition two new cmdletsWait-HXTask and Get-HXTaskStatus are introduced
bull Protected Virtual Machine ScalabilitymdashThis release adds support for 1500 VMs across both clustersand 750 VMs per cluster in a bi-direction or any split between the two clusters without exceeding thelimit of 1500 VMs For more information see Cisco HyperFlex Data Platform Administration Guide
bull System Management REST API enhancementsmdashPause data replication actions briefly using RESTAPI to explicitly inform users on the current status of replication actions
Cisco HyperFlex with Data Platform for Hyper-V
bull Cluster-Aware Updating (CAU)mdashThis is an automated feature that allows you to perform updates onwindows servers in a failover cluster with little or no loss in availability during the upgrade process
Support for new drivesmdashNew drives are qualified for the 40(2a) release The new drives include newcapacity points and new cache drive options Several of the new drives are alternate drives to already qualifiedexisting drives in function which are qualified in 40(2a) These drives are functionally compatible with theexisting drives and are available as alternates in case of lack of availability of existing drives For expansionof existing clusters or general information about interoperability of different drives see Cisco HyperFlexDrive Compatibility
Release Notes for Cisco HX Data Platform Release 406
New Features
NVMe Caching SSDs slot information is unavailable from HX-Connect for all AF server PIDs except forthe All-NVMe server PIDs Please refer to UCSM management console for NVMe SSD slot information
Note
Applicable PlatformsDrive PIDDrive Function
All existing HX M5 serversHX-M2-960GBAlternate boot drive
All existing HX M5 servers except AllNVMe
HX-SD480G6I1X-EVAlternate system (or housekeeping) drive
All existing HX M5 servers except AllNVMe
HX-SD480GM1X-EVAlternate system (or housekeeping) drive
The following HX M5 serversHXAF220C-M5SX HXAF240C-M5SXHXAF-E-220M5SX
HX-SD800G123X-EP800G 12G SAS Cache drive option forAll Flash
All NVMe HXAF220C-M5SNHX-NVME2H-I1000All NVMe 1TB Capacity drive
All NVMe HXAF220C-M5SNHX-NVME2H-I4000All NVMe 4TB Capacity drive
All NVMe HXAF220C-M5SNHX-NVMEHW-I8000New high density All NVMe 8TBCapacity drive
All Flash Configuration ndash namelyHXAF220C-M5SX HXAF240C-M5SXHXAF-E-220M5SX
ESX support only
HX-SD76T61X-EVNew high density All Flash 76TBCapacity drive
Maximum number of drives per node is12 drives on HX AF C240
HX240C-M5LHX-HD8T7K4KANAlternate drive for 8TB LFF capacity
HXAF220C-M5SX HXAF240C-M5SXHX-SD38T2HTNK9New 38TB FIPS compliant SED SSDdata drives
HXAF220C-M5SX HXAF240C-M5SXHX-SD960G2HTNK9New 960GFIPS compliant SEDSSD datadrives
New Features in Release 40(1b)
bull Support for Second Generation Intelreg Xeonreg Scalable Processor RefreshmdashThis release includessupport for the Second Generation Intelreg Xeonreg Scalable processor refresh (formerly Cascade Lake)
New Features in Release 40(1a)
The following new features are in Release 40(1a)
bull Ultra-Light HyperFlex Edge ClustersmdashThis release introduces support for two-node HyperFlex Edgeclusters enabling HyperFlex to run in environments requiring a small footprint Cisco Intersight providescomprehensive lifecycle management and includes remote cloud-based installation centralized upgradesand invisible witnessing Both 1GE and 10GE networking topology options are available
Release Notes for Cisco HX Data Platform Release 407
New Features
bull Scaled-Up HyperFlex Edge ClustersmdashThis release adds support for four node HyperFlex Edge clustersenabling a full range of size options for remote and branch offices Size the branch office environmentsto suit current needs with a two three or four node HyperFlex Edge cluster Cisco Intersight providesfull-lifecycle management and 1GE and 10GE networking options are available
bull Cisco Intersight Invisible Cloud WitnessmdashFor two node clusters this feature eliminates the need forwitness VMs the infrastructure to run those VMs and the management overhead to deploy scale andpatch witnessing software The Invisible Cloud Witness is responsible for maintaining cluster HA in theevent of failure scenarios This feature is included at no additional cost and is automatically deployedand managed by Cisco Intersight
bull Cloud-delivered HyperFlex Edge UpgradesmdashPowered by Cisco Intersight this feature adds supportfor multi-site orchestrated remote upgrades of the HyperFlex Data Platform This feature will be enabledwith the next 40 patch release and will allowHyperFlex Edge clusters deployed via Intersight to performorchestrated upgrades across one or many sites in parallel
bull All-NVMe HyperFlexmdashStarting with this release a new high-end performance node powered by allNVMe drives is available for HyperFlex clusters Co-engineered with Intel to support Intel VMD forhot-plug and surprise removal this offering represents an industry first an enterprise-ready and fullyvalidated all-NVMe HCI appliance The all-NVMe offering is available in the 220 form factor (1RU)and is powered by Intel Optane cache drives for maximum performance and highest endurance
bull VMware Site Recovery Manager (SRM) IntegrationmdashThis release brings support for a Cisco developedStorage Replication Adapter (SRA) for SRM The SRA provides the ability to leverage HyperFlex nativeasync replication with the powerful orchestration and runbook capabilities of SRM The SRA includesthe ability to perform test recoveries planned migrations and full disaster recovery
HX SRA is certified by VMware and is available for download from VMWareSRM site
Note
bull HyperFlex DR Powershell RunbooksmdashNew Powershell cmdlets are included for automated runbookgeneration when using HyperFlex native disaster recovery The New-HXRunbook cmdlet supports thefollowing workflows Test Recovery Planned Migration and Disaster Recovery These runbooks canbe used to orchestrate DR workflows without the requirement for any third-party software
bull Windows Server 2019 with Hyper-VmdashSupport has been added in this release for the Windows Server2019 operating system for Hyper-V based HyperFlex deployments
bull Kubernetes CSI PluginmdashThis release adds support for the HyperFlex CSI (HX-CSI) plugin based onthe Kubernetes Container Storage Interface (CSI) specification Customers can now use the HX-CSIplugin to provision andmanage persistent volumes in Kubernetes version 113 and later Note Kubernetes113 support for Cisco Container Platform amp Openshift Container Platform is forthcoming in theirrespective future releases
bull C480 ML Compute only nodemdashThis release introduces support for C480 ML as a new compute-onlynode for Deep LearningMachine Learning Workloads Data scientists can now use the power of up toeight NVidia SXM2 V100 GPUs to accelerate deep learning workloads VMs running deep-learningworkloads will need to use PCIe pass-through for access to GPUs
bull Higher Capacity DrivesmdashAnew 24TB 10k rpm SAS HDD option for SFF Hybrid HyperFlex clustersand a 12TB 72K rpmSASHDDoption for LFFHybrid are now available Both HyperFlex andHyperFlexEdge support the 24TB capacity point for maximum density in this form factor Note that HyperFlex
Release Notes for Cisco HX Data Platform Release 408
New Features
Edge does not support LFF-drives HyperFlex HyperV version does not yet support the new 12TB driveoption See the HyperFlex spec sheets for a full list of configurable options
bull New Cache and increased scale for Hyper-V mdashNVMe amp Optane SSDs are now supported as cachedrives for Hyper-V deployments Furthermore scale limits have been increased to 16+16(Converged+Compute-only) for both SFF (AF and Hybrid) amp LFF (Hybrid) clusters
bull Centralized Audit Log ExportmdashThis release adds support for audit logging via a remote syslog serverThis capability enables customers to retain audit logs from all HyperFlex nodes in a centralized remotesyslog server to meet retention and compliance requirements
bull DISA STIG CompliancemdashThis release adds new HX REST APIs for setting removing and checkingstatus of DISA STIGs for Controller VMs ESXi hosts and vCenter These APIs enable customers tomeet DISA security requirements by centrally and securely applying STIGs detecting and correctingfor drifts in any STIG settings
Supported Versions and System RequirementsCisco HX Data Platform requires specific software and hardware versions and networking settings forsuccessful installation
For a complete list of requirements see
bull Cisco HyperFlex Systems Installation Guide for VMware ESXi or
bull Cisco HyperFlex Systems Installation Guide for Microsoft Hyper-V
Hardware and Software Interoperability
For a complete list of hardware and software inter-dependencies refer to respective Cisco UCS Managerrelease version of Hardware and Software Interoperability for Cisco HyperFlex HX-Series
HyperFlex Software Versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinan HX Storage Cluster are compatible
bull HyperFlex does not support UCS Manager and UCS Server Firmware versions 40(4a) 40(4b)and 40(4c)
Do not upgrade to these versions of firmware
Do not upgrade to these versions of UCS Manager
Important
bull Verify that the preconfigured HX servers have the same version of Cisco UCS server firmware installedIf the Cisco UCS Fabric Interconnects (FI) firmware versions are different see the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40 for steps to align the firmware versions
bull M4 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M4 or HX220c M4) deploymentsverify that Cisco UCS Manager 31(3k) 32(3i) or 40(2b) or higher is installed For moreinformation see Recommended Cisco HyperFlex HX Data Platform Software Releases
Release Notes for Cisco HX Data Platform Release 409
Supported Versions and System Requirements
bull M5 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M5 or HX220c M5) deploymentsverify that the recommended UCS firmware version is installed
For SED-based HyperFlex systems ensure that the A (Infrastructure) B (Bladeserver) and C (Rack server) bundles are at Cisco UCS Manager version 40(2b)or later for all SED M4M5 systems For more details see CSCvh04307 ForSED-based HyperFlex systems also ensure that all clusters are at HyperFlexRelease 35(2b) or later For more information see Field Notice (70234) andCSCvk17250
Important
bull To reinstall an HX server download supported and compatible versions of the software See theCisco HyperFlex Systems Installation Guide for VMware ESXi for the requirements and steps
bull Important For Intersight edge servers running older than 40(1a) CIMC version HUU is the suggestedmechanism to update the firmware
Table 1 HyperFlex Software Versions for M4M5 Servers
M5 Recommended FIServer Firmware
(be sure to review important notes above)
M4 Recommended FIServer Firmware
(be sure to review important notes above)
HyperFlexRelease
40(4h)40(4h)40(2b)
40(4h)40(4h)40(2a)
40(4h)40(4h)40(1b)
40(4e)40(4e)40(1a)
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Cisco HX Data Platform Release 4x based Deployments
Confirm the component firmware on the server meets the minimum versions listed in the following tables
HyperFlex Edge does not support Cisco IMC versions 40(4a) 40(4b) 40(4c) 40(4d) and 40(4e)Important
Table 2 HX220c M4 HXAF220c M4 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
402d0BIOS
40(2h)Cisco Integrated Management Controller(CIMC)
Release Notes for Cisco HX Data Platform Release 4010
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
13000012SAS HBA
43(2b)VIC 1227
0103 (Toshiba)
CS07 (Intel 480)
SSD
CN03 (Seagate)
CK03 (Seagate 18)
CK03 (Seagate 24)
sF3Q (Samsung)
3F3Q (Samsung)
HDD
40(2h)
Download Software
Host Upgrade Utility (HUU) Download Link
Table 3 HX220c M5 HXAF220c M5 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
C220M5404p00224200755BIOS
40(4i)Cisco Integrated Management Controller(CIMC)
09000008SAS HBA
43(3c)VIC 1387
50(3e)VIC 1457
G201CS01 (Intel)
0104 (Toshiba)
0107 (Toshiba)
SSD
2F3Q (Samsung)
3F3Q (Samsung)
NOA4 (Seagate)
CK03 (Seagate 18tb)
CK03 (Seagate 24)
HDD
Release Notes for Cisco HX Data Platform Release 4011
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
40(4k)
Download Software
Host Upgrade Utility (HUU) Download Link
HyperFlex Licensing
As of version 26(1a) HyperFlex supports VMware PAC licensing Existing VMware embedded licenses willcontinue to be supported
As of version 25(1a) HyperFlex uses a smart licensing mechanism to apply your licenses See the CiscoHyperFlex Systems Installation Guide for VMware ESXi for details and steps
VMware vSphere Licensing Requirements
How you purchase your vSphere license determines how your license applies to your HyperFlex system
bull If you purchased your vSphere license with HyperFlex
Each HyperFlex server either has the Enterprise or Enterprise Plus edition preinstalled at the factory
bull HX Nodes have OEM licenses preinstalled If you delete or overwrite thecontent of the boot drives after receiving the HX servers you also delete thefactory-installed licenses
bull OEM license keys is a newVMware vCenter 60 U1b feature Earlier versionsdo not support OEM licenses
bull All factory-installed HX nodes share the same OEM license key WithvSphere OEM keys the Usage count can exceed the Capacity value
bull When you add an HX host to vCenter through the Add Host wizard in theAssign license section select the OEM license
We obfuscate the actual vSphere OEM license key for example0N085-XXXXX-XXXXX-XXXXX-10LHH
bull Standard Essentials Plus and ROBO editions are not available preinstalledon HX servers
Note
bull If you did NOT purchase your vSphere license with HyperFlex
The HX nodes have a vSphere Foundation license preinstalled After initial setup you can apply thelicense to a supported version of vSphere
bull If you purchased a vSphere PAC license
Follow the instructions in your PAC license letter from VMware to add the license to your MY VMwareaccount then follow the instructions to add your HX host to vCenter and assign the PAC license
Release Notes for Cisco HX Data Platform Release 4012
HyperFlex Licensing
HX Data Platform Software Versions for HyperFlex Witness Node
Witness Node VersionHyperFlex Release
10840(2b)
10840(2a)
10440(1b)
10440(1a)
Software Requirements for VMware ESXi
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and VMware vSphere VMware vCenter and VMware ESXi
bull Verify that all HX servers have a compatible version of vSphere preinstalled
bull Verify that the vCenter version is the same or later than the ESXi version
bull Verify that the vCenter and ESXi versions are compatible by consulting the VMware ProductInteroperability Matrix Newer vCenter versions may be used with older ESXi versions so long as bothESXi and vCenter are supported in the table below
bull Verify that you have a vCenter administrator account with root-level privileges and the associatedpassword
For VIC1457 there is no support for ESXi 60Note
The below table applies for all of the following VMware vSphere Editions Enterprise Enterprise PlusStandard Essentials Plus ROBO
VMware vCenter VersionsVMware ESXi VersionsHyperFlex Version
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2b)1
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2a)2
60 U3 65 U3 67 U260 U3 65 U3 67 U2340(1b)
60 U3 65 U2 67 U260 U3 65 U2 67 U2440(1a)
1 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
2 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
3 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
4 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4013
HX Data Platform Software Versions for HyperFlex Witness Node
Software Requirements for Microsoft Hyper-V
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and Microsoft Hyper-V (Hyper-V) components
HyperFlex Software versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinthe HX Storage Cluster are compatible
bull Cisco HyperFlex M5 Converged nodesmdash For Hybrid (Cisco HyperFlex HX240c M5 HX220c M5)and All Flash (Cisco HyperFlex HXAF240cM5 HXAF220cM5) verify that Cisco UCSManager 40(2b)is installed HX 40 (1a) does not support Hyper-V on the All NVMe (HXAF220C-M5SN) nodes Fordetailed information on installation requirements and steps see theCisco HyperFlex Systems InstallationGuide on Microsoft Hyper-V
Table 4 Supported HyperFlex Software versions for M5 Servers on Hyper-V
M5 Recommended Server FirmwareHyperFlex Release
40(4h)40(2b)
40(4g)40(2a)
40(4e)40(1b)
40(4e)40(1a)
Release Notes for Cisco HX Data Platform Release 4014
Software Requirements for Microsoft Hyper-V
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DescriptionDateRelease
Added CSCvq39523 to the list ofOpen Caveats for Release 40(1a)
July 5 201940(1a)
bull Updated Important Note forSED-based HyperFlexsystems in SupportedVersions and SystemRequirements section
bull Updated Release 35(2b)support for ESXi 67 U2 inSupported VMware vSphereVersions and Editions
July 1 201940(1a)
bull Added CSCvp64140 andCSCvp98910 to the list ofOpen Caveats for Release40(1a)
bull Updated HyperFlex Edge andFirmware CompatibilityMatrix tables
June 20 201940(1a)
Updated BrowserRecommendations
June 17 201940(1a)
Added information indicating40(1a) features are supported onthe Intersight Virtual Appliance andon Intersightcom
May 31 201940(1a)
bull Updated M4M5Recommended FIServerFirmware for 35(2b) and40(1a)
bull Added bullet describingrecommended use of theHypercheck Health CheckUtility in the UpgradeGuidelines section
May 21 201940(1a)
bull Added New Featuredescription for DISA STIGCompliance
bull Updated Storage ClusterSpecifications for Hyper-V
bull Added CSCvp66277 to list ofOpen Caveats
May 14 201940(1a)
Release Notes for Cisco HX Data Platform Release 404
Revision History
DescriptionDateRelease
bull Updated VMware vCenterVersions for 40(1a)
bull Updated SupportedMicrosoftSoftware versions
May 8 201940(1a)
bull Added CSCvo36198 andCSCvk38003 to the list ofResolved Caveats
bull Added CSCvp21417 to the listof Open Caveats
May 3 201940(1a)
Created release notes for Cisco HXData Platform Software Release40(1a)
April 29 201940(1a)
New FeaturesCisco HX Data Platform Release 40 provides the following features These features including the InvisibleCloud Witness for HyperFlex Edge clusters are supported both on the Intersight Virtual Appliance and onIntersightcom
New Features in Release 40(2b)
bull 76TB SSD data drivemdashSupport on HX Edge configurations
bull All NVMe and All Flash limits increasemdashLimits increased on All NVMe (1TB 4TB and 8TB) andAll Flash (76TB SSD)
bull Cluster Scale Limits increasemdashSupport for maximum scale limit increase on a cluster See Cisco HXData Platform Storage Cluster Specifications on page 16
bull HW Offload optionmdashSupport for Hardware Offload option with Stretched cluster configurations
bull Cisco Overlay Transport Virtualization for Stretched ClustermdashSupport for OTV as an overlay forStretched Cluster
New Features in Release 40(2a)
bull Cisco UCS Manager 41(1c)mdashSupport for UCSM hardware and software enhancements and featuresfrom Cisco UCSM 41(1c) release
bull Boost ModemdashThis release introduces Boost Mode for the following configurations All NVMe AllFlash C240 All Flash C220 and Hypervisor ESX Boost Mode allows the Cisco HyperFlex cluster todeliver higher IOPs by increasing the storage controller VMCPU resources by 4 vCPU For configurationinformation see the Cisco HyperFlex Data Platform Administration Guide
bull Cisco HyperFlex HTML plug-in for VMWare vCentermdashEnables virtualization administrator tomanage and monitor the Cisco HyperFlex physical infrastructure by cross launching HyperFlex Connectfrom the vSphere Client UI and perform management actions in the HyperFlex Connect UI
Release Notes for Cisco HX Data Platform Release 405
New Features
bull 25GE networking for HX EdgemdashSupport for 25GE networking for HX Edge
bull All NVMe with Stretched ClustermdashSupport for All NVMe with Stretched Cluster (ESX only)
bull Cluster Upgrade Eligibility TestmdashThis release adds the capability to perform a pre-upgrade test whichchecks for cluster readiness before upgrading Example checks include validating cluster state rebalancestatus controllerVM Free Space ESXi version and much more The Elibility test is intended to helpavoid unexpected problems that may arise during the upgrade process It is highly recommended to runthe test before performing the Hyperflex upgrade
bull Registering Smart Software LicensingmdashThis release adds support for software that allows easy trackingof the status of license and software usage trends and simplifies the three core licensing functionsPurchasing Management and Reporting
bull Dynamic self-signed certificate generation enhancementsmdashThis release adds support for Self-signedSSL certificates on the Controller VMs which were static in prior releases The static certificates arereplaced with dynamically generated self-signed certificates upon upgrading to HXDP 40(2a) so thatthe certificates are unique per cluster The new clusters installed with HXDP 40(2a) have dynamicallygenerated self-signed certificates
bull Test Upgrade EligibilitymdashThis release adds support for testing your cluster readiness and infrastructurecompatibility for an upgrade For more information see the Test Upgrade Eligibility sections in theCiscoHyperFlex SystemsUpgrade Guide for VMware ESXi Release 40 or the CiscoHyperFlex UpgradeGuide for Microsoft Hyper-V Release 40
Disaster Recovery
bull Recovery Settings ConfigurationmdashThis release supports configuration of recovery settings to defineglobal recovery parameters and mapping for resources across recovery sites These parameters are usedduring recovery test recovery and migrate operations
bull HyperFlex DR Powershell RunbookmdashPowerShell runbook functionality is extended to support therecovery configuration settings in the runbooks for all recovery scenarios New-HXrunbook cmdlet cannow be used to generate runbook for a single or multiple protection groups In addition two new cmdletsWait-HXTask and Get-HXTaskStatus are introduced
bull Protected Virtual Machine ScalabilitymdashThis release adds support for 1500 VMs across both clustersand 750 VMs per cluster in a bi-direction or any split between the two clusters without exceeding thelimit of 1500 VMs For more information see Cisco HyperFlex Data Platform Administration Guide
bull System Management REST API enhancementsmdashPause data replication actions briefly using RESTAPI to explicitly inform users on the current status of replication actions
Cisco HyperFlex with Data Platform for Hyper-V
bull Cluster-Aware Updating (CAU)mdashThis is an automated feature that allows you to perform updates onwindows servers in a failover cluster with little or no loss in availability during the upgrade process
Support for new drivesmdashNew drives are qualified for the 40(2a) release The new drives include newcapacity points and new cache drive options Several of the new drives are alternate drives to already qualifiedexisting drives in function which are qualified in 40(2a) These drives are functionally compatible with theexisting drives and are available as alternates in case of lack of availability of existing drives For expansionof existing clusters or general information about interoperability of different drives see Cisco HyperFlexDrive Compatibility
Release Notes for Cisco HX Data Platform Release 406
New Features
NVMe Caching SSDs slot information is unavailable from HX-Connect for all AF server PIDs except forthe All-NVMe server PIDs Please refer to UCSM management console for NVMe SSD slot information
Note
Applicable PlatformsDrive PIDDrive Function
All existing HX M5 serversHX-M2-960GBAlternate boot drive
All existing HX M5 servers except AllNVMe
HX-SD480G6I1X-EVAlternate system (or housekeeping) drive
All existing HX M5 servers except AllNVMe
HX-SD480GM1X-EVAlternate system (or housekeeping) drive
The following HX M5 serversHXAF220C-M5SX HXAF240C-M5SXHXAF-E-220M5SX
HX-SD800G123X-EP800G 12G SAS Cache drive option forAll Flash
All NVMe HXAF220C-M5SNHX-NVME2H-I1000All NVMe 1TB Capacity drive
All NVMe HXAF220C-M5SNHX-NVME2H-I4000All NVMe 4TB Capacity drive
All NVMe HXAF220C-M5SNHX-NVMEHW-I8000New high density All NVMe 8TBCapacity drive
All Flash Configuration ndash namelyHXAF220C-M5SX HXAF240C-M5SXHXAF-E-220M5SX
ESX support only
HX-SD76T61X-EVNew high density All Flash 76TBCapacity drive
Maximum number of drives per node is12 drives on HX AF C240
HX240C-M5LHX-HD8T7K4KANAlternate drive for 8TB LFF capacity
HXAF220C-M5SX HXAF240C-M5SXHX-SD38T2HTNK9New 38TB FIPS compliant SED SSDdata drives
HXAF220C-M5SX HXAF240C-M5SXHX-SD960G2HTNK9New 960GFIPS compliant SEDSSD datadrives
New Features in Release 40(1b)
bull Support for Second Generation Intelreg Xeonreg Scalable Processor RefreshmdashThis release includessupport for the Second Generation Intelreg Xeonreg Scalable processor refresh (formerly Cascade Lake)
New Features in Release 40(1a)
The following new features are in Release 40(1a)
bull Ultra-Light HyperFlex Edge ClustersmdashThis release introduces support for two-node HyperFlex Edgeclusters enabling HyperFlex to run in environments requiring a small footprint Cisco Intersight providescomprehensive lifecycle management and includes remote cloud-based installation centralized upgradesand invisible witnessing Both 1GE and 10GE networking topology options are available
Release Notes for Cisco HX Data Platform Release 407
New Features
bull Scaled-Up HyperFlex Edge ClustersmdashThis release adds support for four node HyperFlex Edge clustersenabling a full range of size options for remote and branch offices Size the branch office environmentsto suit current needs with a two three or four node HyperFlex Edge cluster Cisco Intersight providesfull-lifecycle management and 1GE and 10GE networking options are available
bull Cisco Intersight Invisible Cloud WitnessmdashFor two node clusters this feature eliminates the need forwitness VMs the infrastructure to run those VMs and the management overhead to deploy scale andpatch witnessing software The Invisible Cloud Witness is responsible for maintaining cluster HA in theevent of failure scenarios This feature is included at no additional cost and is automatically deployedand managed by Cisco Intersight
bull Cloud-delivered HyperFlex Edge UpgradesmdashPowered by Cisco Intersight this feature adds supportfor multi-site orchestrated remote upgrades of the HyperFlex Data Platform This feature will be enabledwith the next 40 patch release and will allowHyperFlex Edge clusters deployed via Intersight to performorchestrated upgrades across one or many sites in parallel
bull All-NVMe HyperFlexmdashStarting with this release a new high-end performance node powered by allNVMe drives is available for HyperFlex clusters Co-engineered with Intel to support Intel VMD forhot-plug and surprise removal this offering represents an industry first an enterprise-ready and fullyvalidated all-NVMe HCI appliance The all-NVMe offering is available in the 220 form factor (1RU)and is powered by Intel Optane cache drives for maximum performance and highest endurance
bull VMware Site Recovery Manager (SRM) IntegrationmdashThis release brings support for a Cisco developedStorage Replication Adapter (SRA) for SRM The SRA provides the ability to leverage HyperFlex nativeasync replication with the powerful orchestration and runbook capabilities of SRM The SRA includesthe ability to perform test recoveries planned migrations and full disaster recovery
HX SRA is certified by VMware and is available for download from VMWareSRM site
Note
bull HyperFlex DR Powershell RunbooksmdashNew Powershell cmdlets are included for automated runbookgeneration when using HyperFlex native disaster recovery The New-HXRunbook cmdlet supports thefollowing workflows Test Recovery Planned Migration and Disaster Recovery These runbooks canbe used to orchestrate DR workflows without the requirement for any third-party software
bull Windows Server 2019 with Hyper-VmdashSupport has been added in this release for the Windows Server2019 operating system for Hyper-V based HyperFlex deployments
bull Kubernetes CSI PluginmdashThis release adds support for the HyperFlex CSI (HX-CSI) plugin based onthe Kubernetes Container Storage Interface (CSI) specification Customers can now use the HX-CSIplugin to provision andmanage persistent volumes in Kubernetes version 113 and later Note Kubernetes113 support for Cisco Container Platform amp Openshift Container Platform is forthcoming in theirrespective future releases
bull C480 ML Compute only nodemdashThis release introduces support for C480 ML as a new compute-onlynode for Deep LearningMachine Learning Workloads Data scientists can now use the power of up toeight NVidia SXM2 V100 GPUs to accelerate deep learning workloads VMs running deep-learningworkloads will need to use PCIe pass-through for access to GPUs
bull Higher Capacity DrivesmdashAnew 24TB 10k rpm SAS HDD option for SFF Hybrid HyperFlex clustersand a 12TB 72K rpmSASHDDoption for LFFHybrid are now available Both HyperFlex andHyperFlexEdge support the 24TB capacity point for maximum density in this form factor Note that HyperFlex
Release Notes for Cisco HX Data Platform Release 408
New Features
Edge does not support LFF-drives HyperFlex HyperV version does not yet support the new 12TB driveoption See the HyperFlex spec sheets for a full list of configurable options
bull New Cache and increased scale for Hyper-V mdashNVMe amp Optane SSDs are now supported as cachedrives for Hyper-V deployments Furthermore scale limits have been increased to 16+16(Converged+Compute-only) for both SFF (AF and Hybrid) amp LFF (Hybrid) clusters
bull Centralized Audit Log ExportmdashThis release adds support for audit logging via a remote syslog serverThis capability enables customers to retain audit logs from all HyperFlex nodes in a centralized remotesyslog server to meet retention and compliance requirements
bull DISA STIG CompliancemdashThis release adds new HX REST APIs for setting removing and checkingstatus of DISA STIGs for Controller VMs ESXi hosts and vCenter These APIs enable customers tomeet DISA security requirements by centrally and securely applying STIGs detecting and correctingfor drifts in any STIG settings
Supported Versions and System RequirementsCisco HX Data Platform requires specific software and hardware versions and networking settings forsuccessful installation
For a complete list of requirements see
bull Cisco HyperFlex Systems Installation Guide for VMware ESXi or
bull Cisco HyperFlex Systems Installation Guide for Microsoft Hyper-V
Hardware and Software Interoperability
For a complete list of hardware and software inter-dependencies refer to respective Cisco UCS Managerrelease version of Hardware and Software Interoperability for Cisco HyperFlex HX-Series
HyperFlex Software Versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinan HX Storage Cluster are compatible
bull HyperFlex does not support UCS Manager and UCS Server Firmware versions 40(4a) 40(4b)and 40(4c)
Do not upgrade to these versions of firmware
Do not upgrade to these versions of UCS Manager
Important
bull Verify that the preconfigured HX servers have the same version of Cisco UCS server firmware installedIf the Cisco UCS Fabric Interconnects (FI) firmware versions are different see the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40 for steps to align the firmware versions
bull M4 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M4 or HX220c M4) deploymentsverify that Cisco UCS Manager 31(3k) 32(3i) or 40(2b) or higher is installed For moreinformation see Recommended Cisco HyperFlex HX Data Platform Software Releases
Release Notes for Cisco HX Data Platform Release 409
Supported Versions and System Requirements
bull M5 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M5 or HX220c M5) deploymentsverify that the recommended UCS firmware version is installed
For SED-based HyperFlex systems ensure that the A (Infrastructure) B (Bladeserver) and C (Rack server) bundles are at Cisco UCS Manager version 40(2b)or later for all SED M4M5 systems For more details see CSCvh04307 ForSED-based HyperFlex systems also ensure that all clusters are at HyperFlexRelease 35(2b) or later For more information see Field Notice (70234) andCSCvk17250
Important
bull To reinstall an HX server download supported and compatible versions of the software See theCisco HyperFlex Systems Installation Guide for VMware ESXi for the requirements and steps
bull Important For Intersight edge servers running older than 40(1a) CIMC version HUU is the suggestedmechanism to update the firmware
Table 1 HyperFlex Software Versions for M4M5 Servers
M5 Recommended FIServer Firmware
(be sure to review important notes above)
M4 Recommended FIServer Firmware
(be sure to review important notes above)
HyperFlexRelease
40(4h)40(4h)40(2b)
40(4h)40(4h)40(2a)
40(4h)40(4h)40(1b)
40(4e)40(4e)40(1a)
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Cisco HX Data Platform Release 4x based Deployments
Confirm the component firmware on the server meets the minimum versions listed in the following tables
HyperFlex Edge does not support Cisco IMC versions 40(4a) 40(4b) 40(4c) 40(4d) and 40(4e)Important
Table 2 HX220c M4 HXAF220c M4 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
402d0BIOS
40(2h)Cisco Integrated Management Controller(CIMC)
Release Notes for Cisco HX Data Platform Release 4010
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
13000012SAS HBA
43(2b)VIC 1227
0103 (Toshiba)
CS07 (Intel 480)
SSD
CN03 (Seagate)
CK03 (Seagate 18)
CK03 (Seagate 24)
sF3Q (Samsung)
3F3Q (Samsung)
HDD
40(2h)
Download Software
Host Upgrade Utility (HUU) Download Link
Table 3 HX220c M5 HXAF220c M5 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
C220M5404p00224200755BIOS
40(4i)Cisco Integrated Management Controller(CIMC)
09000008SAS HBA
43(3c)VIC 1387
50(3e)VIC 1457
G201CS01 (Intel)
0104 (Toshiba)
0107 (Toshiba)
SSD
2F3Q (Samsung)
3F3Q (Samsung)
NOA4 (Seagate)
CK03 (Seagate 18tb)
CK03 (Seagate 24)
HDD
Release Notes for Cisco HX Data Platform Release 4011
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
40(4k)
Download Software
Host Upgrade Utility (HUU) Download Link
HyperFlex Licensing
As of version 26(1a) HyperFlex supports VMware PAC licensing Existing VMware embedded licenses willcontinue to be supported
As of version 25(1a) HyperFlex uses a smart licensing mechanism to apply your licenses See the CiscoHyperFlex Systems Installation Guide for VMware ESXi for details and steps
VMware vSphere Licensing Requirements
How you purchase your vSphere license determines how your license applies to your HyperFlex system
bull If you purchased your vSphere license with HyperFlex
Each HyperFlex server either has the Enterprise or Enterprise Plus edition preinstalled at the factory
bull HX Nodes have OEM licenses preinstalled If you delete or overwrite thecontent of the boot drives after receiving the HX servers you also delete thefactory-installed licenses
bull OEM license keys is a newVMware vCenter 60 U1b feature Earlier versionsdo not support OEM licenses
bull All factory-installed HX nodes share the same OEM license key WithvSphere OEM keys the Usage count can exceed the Capacity value
bull When you add an HX host to vCenter through the Add Host wizard in theAssign license section select the OEM license
We obfuscate the actual vSphere OEM license key for example0N085-XXXXX-XXXXX-XXXXX-10LHH
bull Standard Essentials Plus and ROBO editions are not available preinstalledon HX servers
Note
bull If you did NOT purchase your vSphere license with HyperFlex
The HX nodes have a vSphere Foundation license preinstalled After initial setup you can apply thelicense to a supported version of vSphere
bull If you purchased a vSphere PAC license
Follow the instructions in your PAC license letter from VMware to add the license to your MY VMwareaccount then follow the instructions to add your HX host to vCenter and assign the PAC license
Release Notes for Cisco HX Data Platform Release 4012
HyperFlex Licensing
HX Data Platform Software Versions for HyperFlex Witness Node
Witness Node VersionHyperFlex Release
10840(2b)
10840(2a)
10440(1b)
10440(1a)
Software Requirements for VMware ESXi
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and VMware vSphere VMware vCenter and VMware ESXi
bull Verify that all HX servers have a compatible version of vSphere preinstalled
bull Verify that the vCenter version is the same or later than the ESXi version
bull Verify that the vCenter and ESXi versions are compatible by consulting the VMware ProductInteroperability Matrix Newer vCenter versions may be used with older ESXi versions so long as bothESXi and vCenter are supported in the table below
bull Verify that you have a vCenter administrator account with root-level privileges and the associatedpassword
For VIC1457 there is no support for ESXi 60Note
The below table applies for all of the following VMware vSphere Editions Enterprise Enterprise PlusStandard Essentials Plus ROBO
VMware vCenter VersionsVMware ESXi VersionsHyperFlex Version
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2b)1
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2a)2
60 U3 65 U3 67 U260 U3 65 U3 67 U2340(1b)
60 U3 65 U2 67 U260 U3 65 U2 67 U2440(1a)
1 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
2 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
3 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
4 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4013
HX Data Platform Software Versions for HyperFlex Witness Node
Software Requirements for Microsoft Hyper-V
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and Microsoft Hyper-V (Hyper-V) components
HyperFlex Software versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinthe HX Storage Cluster are compatible
bull Cisco HyperFlex M5 Converged nodesmdash For Hybrid (Cisco HyperFlex HX240c M5 HX220c M5)and All Flash (Cisco HyperFlex HXAF240cM5 HXAF220cM5) verify that Cisco UCSManager 40(2b)is installed HX 40 (1a) does not support Hyper-V on the All NVMe (HXAF220C-M5SN) nodes Fordetailed information on installation requirements and steps see theCisco HyperFlex Systems InstallationGuide on Microsoft Hyper-V
Table 4 Supported HyperFlex Software versions for M5 Servers on Hyper-V
M5 Recommended Server FirmwareHyperFlex Release
40(4h)40(2b)
40(4g)40(2a)
40(4e)40(1b)
40(4e)40(1a)
Release Notes for Cisco HX Data Platform Release 4014
Software Requirements for Microsoft Hyper-V
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DescriptionDateRelease
bull Updated VMware vCenterVersions for 40(1a)
bull Updated SupportedMicrosoftSoftware versions
May 8 201940(1a)
bull Added CSCvo36198 andCSCvk38003 to the list ofResolved Caveats
bull Added CSCvp21417 to the listof Open Caveats
May 3 201940(1a)
Created release notes for Cisco HXData Platform Software Release40(1a)
April 29 201940(1a)
New FeaturesCisco HX Data Platform Release 40 provides the following features These features including the InvisibleCloud Witness for HyperFlex Edge clusters are supported both on the Intersight Virtual Appliance and onIntersightcom
New Features in Release 40(2b)
bull 76TB SSD data drivemdashSupport on HX Edge configurations
bull All NVMe and All Flash limits increasemdashLimits increased on All NVMe (1TB 4TB and 8TB) andAll Flash (76TB SSD)
bull Cluster Scale Limits increasemdashSupport for maximum scale limit increase on a cluster See Cisco HXData Platform Storage Cluster Specifications on page 16
bull HW Offload optionmdashSupport for Hardware Offload option with Stretched cluster configurations
bull Cisco Overlay Transport Virtualization for Stretched ClustermdashSupport for OTV as an overlay forStretched Cluster
New Features in Release 40(2a)
bull Cisco UCS Manager 41(1c)mdashSupport for UCSM hardware and software enhancements and featuresfrom Cisco UCSM 41(1c) release
bull Boost ModemdashThis release introduces Boost Mode for the following configurations All NVMe AllFlash C240 All Flash C220 and Hypervisor ESX Boost Mode allows the Cisco HyperFlex cluster todeliver higher IOPs by increasing the storage controller VMCPU resources by 4 vCPU For configurationinformation see the Cisco HyperFlex Data Platform Administration Guide
bull Cisco HyperFlex HTML plug-in for VMWare vCentermdashEnables virtualization administrator tomanage and monitor the Cisco HyperFlex physical infrastructure by cross launching HyperFlex Connectfrom the vSphere Client UI and perform management actions in the HyperFlex Connect UI
Release Notes for Cisco HX Data Platform Release 405
New Features
bull 25GE networking for HX EdgemdashSupport for 25GE networking for HX Edge
bull All NVMe with Stretched ClustermdashSupport for All NVMe with Stretched Cluster (ESX only)
bull Cluster Upgrade Eligibility TestmdashThis release adds the capability to perform a pre-upgrade test whichchecks for cluster readiness before upgrading Example checks include validating cluster state rebalancestatus controllerVM Free Space ESXi version and much more The Elibility test is intended to helpavoid unexpected problems that may arise during the upgrade process It is highly recommended to runthe test before performing the Hyperflex upgrade
bull Registering Smart Software LicensingmdashThis release adds support for software that allows easy trackingof the status of license and software usage trends and simplifies the three core licensing functionsPurchasing Management and Reporting
bull Dynamic self-signed certificate generation enhancementsmdashThis release adds support for Self-signedSSL certificates on the Controller VMs which were static in prior releases The static certificates arereplaced with dynamically generated self-signed certificates upon upgrading to HXDP 40(2a) so thatthe certificates are unique per cluster The new clusters installed with HXDP 40(2a) have dynamicallygenerated self-signed certificates
bull Test Upgrade EligibilitymdashThis release adds support for testing your cluster readiness and infrastructurecompatibility for an upgrade For more information see the Test Upgrade Eligibility sections in theCiscoHyperFlex SystemsUpgrade Guide for VMware ESXi Release 40 or the CiscoHyperFlex UpgradeGuide for Microsoft Hyper-V Release 40
Disaster Recovery
bull Recovery Settings ConfigurationmdashThis release supports configuration of recovery settings to defineglobal recovery parameters and mapping for resources across recovery sites These parameters are usedduring recovery test recovery and migrate operations
bull HyperFlex DR Powershell RunbookmdashPowerShell runbook functionality is extended to support therecovery configuration settings in the runbooks for all recovery scenarios New-HXrunbook cmdlet cannow be used to generate runbook for a single or multiple protection groups In addition two new cmdletsWait-HXTask and Get-HXTaskStatus are introduced
bull Protected Virtual Machine ScalabilitymdashThis release adds support for 1500 VMs across both clustersand 750 VMs per cluster in a bi-direction or any split between the two clusters without exceeding thelimit of 1500 VMs For more information see Cisco HyperFlex Data Platform Administration Guide
bull System Management REST API enhancementsmdashPause data replication actions briefly using RESTAPI to explicitly inform users on the current status of replication actions
Cisco HyperFlex with Data Platform for Hyper-V
bull Cluster-Aware Updating (CAU)mdashThis is an automated feature that allows you to perform updates onwindows servers in a failover cluster with little or no loss in availability during the upgrade process
Support for new drivesmdashNew drives are qualified for the 40(2a) release The new drives include newcapacity points and new cache drive options Several of the new drives are alternate drives to already qualifiedexisting drives in function which are qualified in 40(2a) These drives are functionally compatible with theexisting drives and are available as alternates in case of lack of availability of existing drives For expansionof existing clusters or general information about interoperability of different drives see Cisco HyperFlexDrive Compatibility
Release Notes for Cisco HX Data Platform Release 406
New Features
NVMe Caching SSDs slot information is unavailable from HX-Connect for all AF server PIDs except forthe All-NVMe server PIDs Please refer to UCSM management console for NVMe SSD slot information
Note
Applicable PlatformsDrive PIDDrive Function
All existing HX M5 serversHX-M2-960GBAlternate boot drive
All existing HX M5 servers except AllNVMe
HX-SD480G6I1X-EVAlternate system (or housekeeping) drive
All existing HX M5 servers except AllNVMe
HX-SD480GM1X-EVAlternate system (or housekeeping) drive
The following HX M5 serversHXAF220C-M5SX HXAF240C-M5SXHXAF-E-220M5SX
HX-SD800G123X-EP800G 12G SAS Cache drive option forAll Flash
All NVMe HXAF220C-M5SNHX-NVME2H-I1000All NVMe 1TB Capacity drive
All NVMe HXAF220C-M5SNHX-NVME2H-I4000All NVMe 4TB Capacity drive
All NVMe HXAF220C-M5SNHX-NVMEHW-I8000New high density All NVMe 8TBCapacity drive
All Flash Configuration ndash namelyHXAF220C-M5SX HXAF240C-M5SXHXAF-E-220M5SX
ESX support only
HX-SD76T61X-EVNew high density All Flash 76TBCapacity drive
Maximum number of drives per node is12 drives on HX AF C240
HX240C-M5LHX-HD8T7K4KANAlternate drive for 8TB LFF capacity
HXAF220C-M5SX HXAF240C-M5SXHX-SD38T2HTNK9New 38TB FIPS compliant SED SSDdata drives
HXAF220C-M5SX HXAF240C-M5SXHX-SD960G2HTNK9New 960GFIPS compliant SEDSSD datadrives
New Features in Release 40(1b)
bull Support for Second Generation Intelreg Xeonreg Scalable Processor RefreshmdashThis release includessupport for the Second Generation Intelreg Xeonreg Scalable processor refresh (formerly Cascade Lake)
New Features in Release 40(1a)
The following new features are in Release 40(1a)
bull Ultra-Light HyperFlex Edge ClustersmdashThis release introduces support for two-node HyperFlex Edgeclusters enabling HyperFlex to run in environments requiring a small footprint Cisco Intersight providescomprehensive lifecycle management and includes remote cloud-based installation centralized upgradesand invisible witnessing Both 1GE and 10GE networking topology options are available
Release Notes for Cisco HX Data Platform Release 407
New Features
bull Scaled-Up HyperFlex Edge ClustersmdashThis release adds support for four node HyperFlex Edge clustersenabling a full range of size options for remote and branch offices Size the branch office environmentsto suit current needs with a two three or four node HyperFlex Edge cluster Cisco Intersight providesfull-lifecycle management and 1GE and 10GE networking options are available
bull Cisco Intersight Invisible Cloud WitnessmdashFor two node clusters this feature eliminates the need forwitness VMs the infrastructure to run those VMs and the management overhead to deploy scale andpatch witnessing software The Invisible Cloud Witness is responsible for maintaining cluster HA in theevent of failure scenarios This feature is included at no additional cost and is automatically deployedand managed by Cisco Intersight
bull Cloud-delivered HyperFlex Edge UpgradesmdashPowered by Cisco Intersight this feature adds supportfor multi-site orchestrated remote upgrades of the HyperFlex Data Platform This feature will be enabledwith the next 40 patch release and will allowHyperFlex Edge clusters deployed via Intersight to performorchestrated upgrades across one or many sites in parallel
bull All-NVMe HyperFlexmdashStarting with this release a new high-end performance node powered by allNVMe drives is available for HyperFlex clusters Co-engineered with Intel to support Intel VMD forhot-plug and surprise removal this offering represents an industry first an enterprise-ready and fullyvalidated all-NVMe HCI appliance The all-NVMe offering is available in the 220 form factor (1RU)and is powered by Intel Optane cache drives for maximum performance and highest endurance
bull VMware Site Recovery Manager (SRM) IntegrationmdashThis release brings support for a Cisco developedStorage Replication Adapter (SRA) for SRM The SRA provides the ability to leverage HyperFlex nativeasync replication with the powerful orchestration and runbook capabilities of SRM The SRA includesthe ability to perform test recoveries planned migrations and full disaster recovery
HX SRA is certified by VMware and is available for download from VMWareSRM site
Note
bull HyperFlex DR Powershell RunbooksmdashNew Powershell cmdlets are included for automated runbookgeneration when using HyperFlex native disaster recovery The New-HXRunbook cmdlet supports thefollowing workflows Test Recovery Planned Migration and Disaster Recovery These runbooks canbe used to orchestrate DR workflows without the requirement for any third-party software
bull Windows Server 2019 with Hyper-VmdashSupport has been added in this release for the Windows Server2019 operating system for Hyper-V based HyperFlex deployments
bull Kubernetes CSI PluginmdashThis release adds support for the HyperFlex CSI (HX-CSI) plugin based onthe Kubernetes Container Storage Interface (CSI) specification Customers can now use the HX-CSIplugin to provision andmanage persistent volumes in Kubernetes version 113 and later Note Kubernetes113 support for Cisco Container Platform amp Openshift Container Platform is forthcoming in theirrespective future releases
bull C480 ML Compute only nodemdashThis release introduces support for C480 ML as a new compute-onlynode for Deep LearningMachine Learning Workloads Data scientists can now use the power of up toeight NVidia SXM2 V100 GPUs to accelerate deep learning workloads VMs running deep-learningworkloads will need to use PCIe pass-through for access to GPUs
bull Higher Capacity DrivesmdashAnew 24TB 10k rpm SAS HDD option for SFF Hybrid HyperFlex clustersand a 12TB 72K rpmSASHDDoption for LFFHybrid are now available Both HyperFlex andHyperFlexEdge support the 24TB capacity point for maximum density in this form factor Note that HyperFlex
Release Notes for Cisco HX Data Platform Release 408
New Features
Edge does not support LFF-drives HyperFlex HyperV version does not yet support the new 12TB driveoption See the HyperFlex spec sheets for a full list of configurable options
bull New Cache and increased scale for Hyper-V mdashNVMe amp Optane SSDs are now supported as cachedrives for Hyper-V deployments Furthermore scale limits have been increased to 16+16(Converged+Compute-only) for both SFF (AF and Hybrid) amp LFF (Hybrid) clusters
bull Centralized Audit Log ExportmdashThis release adds support for audit logging via a remote syslog serverThis capability enables customers to retain audit logs from all HyperFlex nodes in a centralized remotesyslog server to meet retention and compliance requirements
bull DISA STIG CompliancemdashThis release adds new HX REST APIs for setting removing and checkingstatus of DISA STIGs for Controller VMs ESXi hosts and vCenter These APIs enable customers tomeet DISA security requirements by centrally and securely applying STIGs detecting and correctingfor drifts in any STIG settings
Supported Versions and System RequirementsCisco HX Data Platform requires specific software and hardware versions and networking settings forsuccessful installation
For a complete list of requirements see
bull Cisco HyperFlex Systems Installation Guide for VMware ESXi or
bull Cisco HyperFlex Systems Installation Guide for Microsoft Hyper-V
Hardware and Software Interoperability
For a complete list of hardware and software inter-dependencies refer to respective Cisco UCS Managerrelease version of Hardware and Software Interoperability for Cisco HyperFlex HX-Series
HyperFlex Software Versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinan HX Storage Cluster are compatible
bull HyperFlex does not support UCS Manager and UCS Server Firmware versions 40(4a) 40(4b)and 40(4c)
Do not upgrade to these versions of firmware
Do not upgrade to these versions of UCS Manager
Important
bull Verify that the preconfigured HX servers have the same version of Cisco UCS server firmware installedIf the Cisco UCS Fabric Interconnects (FI) firmware versions are different see the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40 for steps to align the firmware versions
bull M4 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M4 or HX220c M4) deploymentsverify that Cisco UCS Manager 31(3k) 32(3i) or 40(2b) or higher is installed For moreinformation see Recommended Cisco HyperFlex HX Data Platform Software Releases
Release Notes for Cisco HX Data Platform Release 409
Supported Versions and System Requirements
bull M5 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M5 or HX220c M5) deploymentsverify that the recommended UCS firmware version is installed
For SED-based HyperFlex systems ensure that the A (Infrastructure) B (Bladeserver) and C (Rack server) bundles are at Cisco UCS Manager version 40(2b)or later for all SED M4M5 systems For more details see CSCvh04307 ForSED-based HyperFlex systems also ensure that all clusters are at HyperFlexRelease 35(2b) or later For more information see Field Notice (70234) andCSCvk17250
Important
bull To reinstall an HX server download supported and compatible versions of the software See theCisco HyperFlex Systems Installation Guide for VMware ESXi for the requirements and steps
bull Important For Intersight edge servers running older than 40(1a) CIMC version HUU is the suggestedmechanism to update the firmware
Table 1 HyperFlex Software Versions for M4M5 Servers
M5 Recommended FIServer Firmware
(be sure to review important notes above)
M4 Recommended FIServer Firmware
(be sure to review important notes above)
HyperFlexRelease
40(4h)40(4h)40(2b)
40(4h)40(4h)40(2a)
40(4h)40(4h)40(1b)
40(4e)40(4e)40(1a)
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Cisco HX Data Platform Release 4x based Deployments
Confirm the component firmware on the server meets the minimum versions listed in the following tables
HyperFlex Edge does not support Cisco IMC versions 40(4a) 40(4b) 40(4c) 40(4d) and 40(4e)Important
Table 2 HX220c M4 HXAF220c M4 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
402d0BIOS
40(2h)Cisco Integrated Management Controller(CIMC)
Release Notes for Cisco HX Data Platform Release 4010
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
13000012SAS HBA
43(2b)VIC 1227
0103 (Toshiba)
CS07 (Intel 480)
SSD
CN03 (Seagate)
CK03 (Seagate 18)
CK03 (Seagate 24)
sF3Q (Samsung)
3F3Q (Samsung)
HDD
40(2h)
Download Software
Host Upgrade Utility (HUU) Download Link
Table 3 HX220c M5 HXAF220c M5 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
C220M5404p00224200755BIOS
40(4i)Cisco Integrated Management Controller(CIMC)
09000008SAS HBA
43(3c)VIC 1387
50(3e)VIC 1457
G201CS01 (Intel)
0104 (Toshiba)
0107 (Toshiba)
SSD
2F3Q (Samsung)
3F3Q (Samsung)
NOA4 (Seagate)
CK03 (Seagate 18tb)
CK03 (Seagate 24)
HDD
Release Notes for Cisco HX Data Platform Release 4011
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
40(4k)
Download Software
Host Upgrade Utility (HUU) Download Link
HyperFlex Licensing
As of version 26(1a) HyperFlex supports VMware PAC licensing Existing VMware embedded licenses willcontinue to be supported
As of version 25(1a) HyperFlex uses a smart licensing mechanism to apply your licenses See the CiscoHyperFlex Systems Installation Guide for VMware ESXi for details and steps
VMware vSphere Licensing Requirements
How you purchase your vSphere license determines how your license applies to your HyperFlex system
bull If you purchased your vSphere license with HyperFlex
Each HyperFlex server either has the Enterprise or Enterprise Plus edition preinstalled at the factory
bull HX Nodes have OEM licenses preinstalled If you delete or overwrite thecontent of the boot drives after receiving the HX servers you also delete thefactory-installed licenses
bull OEM license keys is a newVMware vCenter 60 U1b feature Earlier versionsdo not support OEM licenses
bull All factory-installed HX nodes share the same OEM license key WithvSphere OEM keys the Usage count can exceed the Capacity value
bull When you add an HX host to vCenter through the Add Host wizard in theAssign license section select the OEM license
We obfuscate the actual vSphere OEM license key for example0N085-XXXXX-XXXXX-XXXXX-10LHH
bull Standard Essentials Plus and ROBO editions are not available preinstalledon HX servers
Note
bull If you did NOT purchase your vSphere license with HyperFlex
The HX nodes have a vSphere Foundation license preinstalled After initial setup you can apply thelicense to a supported version of vSphere
bull If you purchased a vSphere PAC license
Follow the instructions in your PAC license letter from VMware to add the license to your MY VMwareaccount then follow the instructions to add your HX host to vCenter and assign the PAC license
Release Notes for Cisco HX Data Platform Release 4012
HyperFlex Licensing
HX Data Platform Software Versions for HyperFlex Witness Node
Witness Node VersionHyperFlex Release
10840(2b)
10840(2a)
10440(1b)
10440(1a)
Software Requirements for VMware ESXi
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and VMware vSphere VMware vCenter and VMware ESXi
bull Verify that all HX servers have a compatible version of vSphere preinstalled
bull Verify that the vCenter version is the same or later than the ESXi version
bull Verify that the vCenter and ESXi versions are compatible by consulting the VMware ProductInteroperability Matrix Newer vCenter versions may be used with older ESXi versions so long as bothESXi and vCenter are supported in the table below
bull Verify that you have a vCenter administrator account with root-level privileges and the associatedpassword
For VIC1457 there is no support for ESXi 60Note
The below table applies for all of the following VMware vSphere Editions Enterprise Enterprise PlusStandard Essentials Plus ROBO
VMware vCenter VersionsVMware ESXi VersionsHyperFlex Version
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2b)1
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2a)2
60 U3 65 U3 67 U260 U3 65 U3 67 U2340(1b)
60 U3 65 U2 67 U260 U3 65 U2 67 U2440(1a)
1 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
2 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
3 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
4 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4013
HX Data Platform Software Versions for HyperFlex Witness Node
Software Requirements for Microsoft Hyper-V
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and Microsoft Hyper-V (Hyper-V) components
HyperFlex Software versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinthe HX Storage Cluster are compatible
bull Cisco HyperFlex M5 Converged nodesmdash For Hybrid (Cisco HyperFlex HX240c M5 HX220c M5)and All Flash (Cisco HyperFlex HXAF240cM5 HXAF220cM5) verify that Cisco UCSManager 40(2b)is installed HX 40 (1a) does not support Hyper-V on the All NVMe (HXAF220C-M5SN) nodes Fordetailed information on installation requirements and steps see theCisco HyperFlex Systems InstallationGuide on Microsoft Hyper-V
Table 4 Supported HyperFlex Software versions for M5 Servers on Hyper-V
M5 Recommended Server FirmwareHyperFlex Release
40(4h)40(2b)
40(4g)40(2a)
40(4e)40(1b)
40(4e)40(1a)
Release Notes for Cisco HX Data Platform Release 4014
Software Requirements for Microsoft Hyper-V
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
bull 25GE networking for HX EdgemdashSupport for 25GE networking for HX Edge
bull All NVMe with Stretched ClustermdashSupport for All NVMe with Stretched Cluster (ESX only)
bull Cluster Upgrade Eligibility TestmdashThis release adds the capability to perform a pre-upgrade test whichchecks for cluster readiness before upgrading Example checks include validating cluster state rebalancestatus controllerVM Free Space ESXi version and much more The Elibility test is intended to helpavoid unexpected problems that may arise during the upgrade process It is highly recommended to runthe test before performing the Hyperflex upgrade
bull Registering Smart Software LicensingmdashThis release adds support for software that allows easy trackingof the status of license and software usage trends and simplifies the three core licensing functionsPurchasing Management and Reporting
bull Dynamic self-signed certificate generation enhancementsmdashThis release adds support for Self-signedSSL certificates on the Controller VMs which were static in prior releases The static certificates arereplaced with dynamically generated self-signed certificates upon upgrading to HXDP 40(2a) so thatthe certificates are unique per cluster The new clusters installed with HXDP 40(2a) have dynamicallygenerated self-signed certificates
bull Test Upgrade EligibilitymdashThis release adds support for testing your cluster readiness and infrastructurecompatibility for an upgrade For more information see the Test Upgrade Eligibility sections in theCiscoHyperFlex SystemsUpgrade Guide for VMware ESXi Release 40 or the CiscoHyperFlex UpgradeGuide for Microsoft Hyper-V Release 40
Disaster Recovery
bull Recovery Settings ConfigurationmdashThis release supports configuration of recovery settings to defineglobal recovery parameters and mapping for resources across recovery sites These parameters are usedduring recovery test recovery and migrate operations
bull HyperFlex DR Powershell RunbookmdashPowerShell runbook functionality is extended to support therecovery configuration settings in the runbooks for all recovery scenarios New-HXrunbook cmdlet cannow be used to generate runbook for a single or multiple protection groups In addition two new cmdletsWait-HXTask and Get-HXTaskStatus are introduced
bull Protected Virtual Machine ScalabilitymdashThis release adds support for 1500 VMs across both clustersand 750 VMs per cluster in a bi-direction or any split between the two clusters without exceeding thelimit of 1500 VMs For more information see Cisco HyperFlex Data Platform Administration Guide
bull System Management REST API enhancementsmdashPause data replication actions briefly using RESTAPI to explicitly inform users on the current status of replication actions
Cisco HyperFlex with Data Platform for Hyper-V
bull Cluster-Aware Updating (CAU)mdashThis is an automated feature that allows you to perform updates onwindows servers in a failover cluster with little or no loss in availability during the upgrade process
Support for new drivesmdashNew drives are qualified for the 40(2a) release The new drives include newcapacity points and new cache drive options Several of the new drives are alternate drives to already qualifiedexisting drives in function which are qualified in 40(2a) These drives are functionally compatible with theexisting drives and are available as alternates in case of lack of availability of existing drives For expansionof existing clusters or general information about interoperability of different drives see Cisco HyperFlexDrive Compatibility
Release Notes for Cisco HX Data Platform Release 406
New Features
NVMe Caching SSDs slot information is unavailable from HX-Connect for all AF server PIDs except forthe All-NVMe server PIDs Please refer to UCSM management console for NVMe SSD slot information
Note
Applicable PlatformsDrive PIDDrive Function
All existing HX M5 serversHX-M2-960GBAlternate boot drive
All existing HX M5 servers except AllNVMe
HX-SD480G6I1X-EVAlternate system (or housekeeping) drive
All existing HX M5 servers except AllNVMe
HX-SD480GM1X-EVAlternate system (or housekeeping) drive
The following HX M5 serversHXAF220C-M5SX HXAF240C-M5SXHXAF-E-220M5SX
HX-SD800G123X-EP800G 12G SAS Cache drive option forAll Flash
All NVMe HXAF220C-M5SNHX-NVME2H-I1000All NVMe 1TB Capacity drive
All NVMe HXAF220C-M5SNHX-NVME2H-I4000All NVMe 4TB Capacity drive
All NVMe HXAF220C-M5SNHX-NVMEHW-I8000New high density All NVMe 8TBCapacity drive
All Flash Configuration ndash namelyHXAF220C-M5SX HXAF240C-M5SXHXAF-E-220M5SX
ESX support only
HX-SD76T61X-EVNew high density All Flash 76TBCapacity drive
Maximum number of drives per node is12 drives on HX AF C240
HX240C-M5LHX-HD8T7K4KANAlternate drive for 8TB LFF capacity
HXAF220C-M5SX HXAF240C-M5SXHX-SD38T2HTNK9New 38TB FIPS compliant SED SSDdata drives
HXAF220C-M5SX HXAF240C-M5SXHX-SD960G2HTNK9New 960GFIPS compliant SEDSSD datadrives
New Features in Release 40(1b)
bull Support for Second Generation Intelreg Xeonreg Scalable Processor RefreshmdashThis release includessupport for the Second Generation Intelreg Xeonreg Scalable processor refresh (formerly Cascade Lake)
New Features in Release 40(1a)
The following new features are in Release 40(1a)
bull Ultra-Light HyperFlex Edge ClustersmdashThis release introduces support for two-node HyperFlex Edgeclusters enabling HyperFlex to run in environments requiring a small footprint Cisco Intersight providescomprehensive lifecycle management and includes remote cloud-based installation centralized upgradesand invisible witnessing Both 1GE and 10GE networking topology options are available
Release Notes for Cisco HX Data Platform Release 407
New Features
bull Scaled-Up HyperFlex Edge ClustersmdashThis release adds support for four node HyperFlex Edge clustersenabling a full range of size options for remote and branch offices Size the branch office environmentsto suit current needs with a two three or four node HyperFlex Edge cluster Cisco Intersight providesfull-lifecycle management and 1GE and 10GE networking options are available
bull Cisco Intersight Invisible Cloud WitnessmdashFor two node clusters this feature eliminates the need forwitness VMs the infrastructure to run those VMs and the management overhead to deploy scale andpatch witnessing software The Invisible Cloud Witness is responsible for maintaining cluster HA in theevent of failure scenarios This feature is included at no additional cost and is automatically deployedand managed by Cisco Intersight
bull Cloud-delivered HyperFlex Edge UpgradesmdashPowered by Cisco Intersight this feature adds supportfor multi-site orchestrated remote upgrades of the HyperFlex Data Platform This feature will be enabledwith the next 40 patch release and will allowHyperFlex Edge clusters deployed via Intersight to performorchestrated upgrades across one or many sites in parallel
bull All-NVMe HyperFlexmdashStarting with this release a new high-end performance node powered by allNVMe drives is available for HyperFlex clusters Co-engineered with Intel to support Intel VMD forhot-plug and surprise removal this offering represents an industry first an enterprise-ready and fullyvalidated all-NVMe HCI appliance The all-NVMe offering is available in the 220 form factor (1RU)and is powered by Intel Optane cache drives for maximum performance and highest endurance
bull VMware Site Recovery Manager (SRM) IntegrationmdashThis release brings support for a Cisco developedStorage Replication Adapter (SRA) for SRM The SRA provides the ability to leverage HyperFlex nativeasync replication with the powerful orchestration and runbook capabilities of SRM The SRA includesthe ability to perform test recoveries planned migrations and full disaster recovery
HX SRA is certified by VMware and is available for download from VMWareSRM site
Note
bull HyperFlex DR Powershell RunbooksmdashNew Powershell cmdlets are included for automated runbookgeneration when using HyperFlex native disaster recovery The New-HXRunbook cmdlet supports thefollowing workflows Test Recovery Planned Migration and Disaster Recovery These runbooks canbe used to orchestrate DR workflows without the requirement for any third-party software
bull Windows Server 2019 with Hyper-VmdashSupport has been added in this release for the Windows Server2019 operating system for Hyper-V based HyperFlex deployments
bull Kubernetes CSI PluginmdashThis release adds support for the HyperFlex CSI (HX-CSI) plugin based onthe Kubernetes Container Storage Interface (CSI) specification Customers can now use the HX-CSIplugin to provision andmanage persistent volumes in Kubernetes version 113 and later Note Kubernetes113 support for Cisco Container Platform amp Openshift Container Platform is forthcoming in theirrespective future releases
bull C480 ML Compute only nodemdashThis release introduces support for C480 ML as a new compute-onlynode for Deep LearningMachine Learning Workloads Data scientists can now use the power of up toeight NVidia SXM2 V100 GPUs to accelerate deep learning workloads VMs running deep-learningworkloads will need to use PCIe pass-through for access to GPUs
bull Higher Capacity DrivesmdashAnew 24TB 10k rpm SAS HDD option for SFF Hybrid HyperFlex clustersand a 12TB 72K rpmSASHDDoption for LFFHybrid are now available Both HyperFlex andHyperFlexEdge support the 24TB capacity point for maximum density in this form factor Note that HyperFlex
Release Notes for Cisco HX Data Platform Release 408
New Features
Edge does not support LFF-drives HyperFlex HyperV version does not yet support the new 12TB driveoption See the HyperFlex spec sheets for a full list of configurable options
bull New Cache and increased scale for Hyper-V mdashNVMe amp Optane SSDs are now supported as cachedrives for Hyper-V deployments Furthermore scale limits have been increased to 16+16(Converged+Compute-only) for both SFF (AF and Hybrid) amp LFF (Hybrid) clusters
bull Centralized Audit Log ExportmdashThis release adds support for audit logging via a remote syslog serverThis capability enables customers to retain audit logs from all HyperFlex nodes in a centralized remotesyslog server to meet retention and compliance requirements
bull DISA STIG CompliancemdashThis release adds new HX REST APIs for setting removing and checkingstatus of DISA STIGs for Controller VMs ESXi hosts and vCenter These APIs enable customers tomeet DISA security requirements by centrally and securely applying STIGs detecting and correctingfor drifts in any STIG settings
Supported Versions and System RequirementsCisco HX Data Platform requires specific software and hardware versions and networking settings forsuccessful installation
For a complete list of requirements see
bull Cisco HyperFlex Systems Installation Guide for VMware ESXi or
bull Cisco HyperFlex Systems Installation Guide for Microsoft Hyper-V
Hardware and Software Interoperability
For a complete list of hardware and software inter-dependencies refer to respective Cisco UCS Managerrelease version of Hardware and Software Interoperability for Cisco HyperFlex HX-Series
HyperFlex Software Versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinan HX Storage Cluster are compatible
bull HyperFlex does not support UCS Manager and UCS Server Firmware versions 40(4a) 40(4b)and 40(4c)
Do not upgrade to these versions of firmware
Do not upgrade to these versions of UCS Manager
Important
bull Verify that the preconfigured HX servers have the same version of Cisco UCS server firmware installedIf the Cisco UCS Fabric Interconnects (FI) firmware versions are different see the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40 for steps to align the firmware versions
bull M4 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M4 or HX220c M4) deploymentsverify that Cisco UCS Manager 31(3k) 32(3i) or 40(2b) or higher is installed For moreinformation see Recommended Cisco HyperFlex HX Data Platform Software Releases
Release Notes for Cisco HX Data Platform Release 409
Supported Versions and System Requirements
bull M5 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M5 or HX220c M5) deploymentsverify that the recommended UCS firmware version is installed
For SED-based HyperFlex systems ensure that the A (Infrastructure) B (Bladeserver) and C (Rack server) bundles are at Cisco UCS Manager version 40(2b)or later for all SED M4M5 systems For more details see CSCvh04307 ForSED-based HyperFlex systems also ensure that all clusters are at HyperFlexRelease 35(2b) or later For more information see Field Notice (70234) andCSCvk17250
Important
bull To reinstall an HX server download supported and compatible versions of the software See theCisco HyperFlex Systems Installation Guide for VMware ESXi for the requirements and steps
bull Important For Intersight edge servers running older than 40(1a) CIMC version HUU is the suggestedmechanism to update the firmware
Table 1 HyperFlex Software Versions for M4M5 Servers
M5 Recommended FIServer Firmware
(be sure to review important notes above)
M4 Recommended FIServer Firmware
(be sure to review important notes above)
HyperFlexRelease
40(4h)40(4h)40(2b)
40(4h)40(4h)40(2a)
40(4h)40(4h)40(1b)
40(4e)40(4e)40(1a)
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Cisco HX Data Platform Release 4x based Deployments
Confirm the component firmware on the server meets the minimum versions listed in the following tables
HyperFlex Edge does not support Cisco IMC versions 40(4a) 40(4b) 40(4c) 40(4d) and 40(4e)Important
Table 2 HX220c M4 HXAF220c M4 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
402d0BIOS
40(2h)Cisco Integrated Management Controller(CIMC)
Release Notes for Cisco HX Data Platform Release 4010
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
13000012SAS HBA
43(2b)VIC 1227
0103 (Toshiba)
CS07 (Intel 480)
SSD
CN03 (Seagate)
CK03 (Seagate 18)
CK03 (Seagate 24)
sF3Q (Samsung)
3F3Q (Samsung)
HDD
40(2h)
Download Software
Host Upgrade Utility (HUU) Download Link
Table 3 HX220c M5 HXAF220c M5 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
C220M5404p00224200755BIOS
40(4i)Cisco Integrated Management Controller(CIMC)
09000008SAS HBA
43(3c)VIC 1387
50(3e)VIC 1457
G201CS01 (Intel)
0104 (Toshiba)
0107 (Toshiba)
SSD
2F3Q (Samsung)
3F3Q (Samsung)
NOA4 (Seagate)
CK03 (Seagate 18tb)
CK03 (Seagate 24)
HDD
Release Notes for Cisco HX Data Platform Release 4011
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
40(4k)
Download Software
Host Upgrade Utility (HUU) Download Link
HyperFlex Licensing
As of version 26(1a) HyperFlex supports VMware PAC licensing Existing VMware embedded licenses willcontinue to be supported
As of version 25(1a) HyperFlex uses a smart licensing mechanism to apply your licenses See the CiscoHyperFlex Systems Installation Guide for VMware ESXi for details and steps
VMware vSphere Licensing Requirements
How you purchase your vSphere license determines how your license applies to your HyperFlex system
bull If you purchased your vSphere license with HyperFlex
Each HyperFlex server either has the Enterprise or Enterprise Plus edition preinstalled at the factory
bull HX Nodes have OEM licenses preinstalled If you delete or overwrite thecontent of the boot drives after receiving the HX servers you also delete thefactory-installed licenses
bull OEM license keys is a newVMware vCenter 60 U1b feature Earlier versionsdo not support OEM licenses
bull All factory-installed HX nodes share the same OEM license key WithvSphere OEM keys the Usage count can exceed the Capacity value
bull When you add an HX host to vCenter through the Add Host wizard in theAssign license section select the OEM license
We obfuscate the actual vSphere OEM license key for example0N085-XXXXX-XXXXX-XXXXX-10LHH
bull Standard Essentials Plus and ROBO editions are not available preinstalledon HX servers
Note
bull If you did NOT purchase your vSphere license with HyperFlex
The HX nodes have a vSphere Foundation license preinstalled After initial setup you can apply thelicense to a supported version of vSphere
bull If you purchased a vSphere PAC license
Follow the instructions in your PAC license letter from VMware to add the license to your MY VMwareaccount then follow the instructions to add your HX host to vCenter and assign the PAC license
Release Notes for Cisco HX Data Platform Release 4012
HyperFlex Licensing
HX Data Platform Software Versions for HyperFlex Witness Node
Witness Node VersionHyperFlex Release
10840(2b)
10840(2a)
10440(1b)
10440(1a)
Software Requirements for VMware ESXi
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and VMware vSphere VMware vCenter and VMware ESXi
bull Verify that all HX servers have a compatible version of vSphere preinstalled
bull Verify that the vCenter version is the same or later than the ESXi version
bull Verify that the vCenter and ESXi versions are compatible by consulting the VMware ProductInteroperability Matrix Newer vCenter versions may be used with older ESXi versions so long as bothESXi and vCenter are supported in the table below
bull Verify that you have a vCenter administrator account with root-level privileges and the associatedpassword
For VIC1457 there is no support for ESXi 60Note
The below table applies for all of the following VMware vSphere Editions Enterprise Enterprise PlusStandard Essentials Plus ROBO
VMware vCenter VersionsVMware ESXi VersionsHyperFlex Version
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2b)1
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2a)2
60 U3 65 U3 67 U260 U3 65 U3 67 U2340(1b)
60 U3 65 U2 67 U260 U3 65 U2 67 U2440(1a)
1 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
2 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
3 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
4 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4013
HX Data Platform Software Versions for HyperFlex Witness Node
Software Requirements for Microsoft Hyper-V
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and Microsoft Hyper-V (Hyper-V) components
HyperFlex Software versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinthe HX Storage Cluster are compatible
bull Cisco HyperFlex M5 Converged nodesmdash For Hybrid (Cisco HyperFlex HX240c M5 HX220c M5)and All Flash (Cisco HyperFlex HXAF240cM5 HXAF220cM5) verify that Cisco UCSManager 40(2b)is installed HX 40 (1a) does not support Hyper-V on the All NVMe (HXAF220C-M5SN) nodes Fordetailed information on installation requirements and steps see theCisco HyperFlex Systems InstallationGuide on Microsoft Hyper-V
Table 4 Supported HyperFlex Software versions for M5 Servers on Hyper-V
M5 Recommended Server FirmwareHyperFlex Release
40(4h)40(2b)
40(4g)40(2a)
40(4e)40(1b)
40(4e)40(1a)
Release Notes for Cisco HX Data Platform Release 4014
Software Requirements for Microsoft Hyper-V
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
NVMe Caching SSDs slot information is unavailable from HX-Connect for all AF server PIDs except forthe All-NVMe server PIDs Please refer to UCSM management console for NVMe SSD slot information
Note
Applicable PlatformsDrive PIDDrive Function
All existing HX M5 serversHX-M2-960GBAlternate boot drive
All existing HX M5 servers except AllNVMe
HX-SD480G6I1X-EVAlternate system (or housekeeping) drive
All existing HX M5 servers except AllNVMe
HX-SD480GM1X-EVAlternate system (or housekeeping) drive
The following HX M5 serversHXAF220C-M5SX HXAF240C-M5SXHXAF-E-220M5SX
HX-SD800G123X-EP800G 12G SAS Cache drive option forAll Flash
All NVMe HXAF220C-M5SNHX-NVME2H-I1000All NVMe 1TB Capacity drive
All NVMe HXAF220C-M5SNHX-NVME2H-I4000All NVMe 4TB Capacity drive
All NVMe HXAF220C-M5SNHX-NVMEHW-I8000New high density All NVMe 8TBCapacity drive
All Flash Configuration ndash namelyHXAF220C-M5SX HXAF240C-M5SXHXAF-E-220M5SX
ESX support only
HX-SD76T61X-EVNew high density All Flash 76TBCapacity drive
Maximum number of drives per node is12 drives on HX AF C240
HX240C-M5LHX-HD8T7K4KANAlternate drive for 8TB LFF capacity
HXAF220C-M5SX HXAF240C-M5SXHX-SD38T2HTNK9New 38TB FIPS compliant SED SSDdata drives
HXAF220C-M5SX HXAF240C-M5SXHX-SD960G2HTNK9New 960GFIPS compliant SEDSSD datadrives
New Features in Release 40(1b)
bull Support for Second Generation Intelreg Xeonreg Scalable Processor RefreshmdashThis release includessupport for the Second Generation Intelreg Xeonreg Scalable processor refresh (formerly Cascade Lake)
New Features in Release 40(1a)
The following new features are in Release 40(1a)
bull Ultra-Light HyperFlex Edge ClustersmdashThis release introduces support for two-node HyperFlex Edgeclusters enabling HyperFlex to run in environments requiring a small footprint Cisco Intersight providescomprehensive lifecycle management and includes remote cloud-based installation centralized upgradesand invisible witnessing Both 1GE and 10GE networking topology options are available
Release Notes for Cisco HX Data Platform Release 407
New Features
bull Scaled-Up HyperFlex Edge ClustersmdashThis release adds support for four node HyperFlex Edge clustersenabling a full range of size options for remote and branch offices Size the branch office environmentsto suit current needs with a two three or four node HyperFlex Edge cluster Cisco Intersight providesfull-lifecycle management and 1GE and 10GE networking options are available
bull Cisco Intersight Invisible Cloud WitnessmdashFor two node clusters this feature eliminates the need forwitness VMs the infrastructure to run those VMs and the management overhead to deploy scale andpatch witnessing software The Invisible Cloud Witness is responsible for maintaining cluster HA in theevent of failure scenarios This feature is included at no additional cost and is automatically deployedand managed by Cisco Intersight
bull Cloud-delivered HyperFlex Edge UpgradesmdashPowered by Cisco Intersight this feature adds supportfor multi-site orchestrated remote upgrades of the HyperFlex Data Platform This feature will be enabledwith the next 40 patch release and will allowHyperFlex Edge clusters deployed via Intersight to performorchestrated upgrades across one or many sites in parallel
bull All-NVMe HyperFlexmdashStarting with this release a new high-end performance node powered by allNVMe drives is available for HyperFlex clusters Co-engineered with Intel to support Intel VMD forhot-plug and surprise removal this offering represents an industry first an enterprise-ready and fullyvalidated all-NVMe HCI appliance The all-NVMe offering is available in the 220 form factor (1RU)and is powered by Intel Optane cache drives for maximum performance and highest endurance
bull VMware Site Recovery Manager (SRM) IntegrationmdashThis release brings support for a Cisco developedStorage Replication Adapter (SRA) for SRM The SRA provides the ability to leverage HyperFlex nativeasync replication with the powerful orchestration and runbook capabilities of SRM The SRA includesthe ability to perform test recoveries planned migrations and full disaster recovery
HX SRA is certified by VMware and is available for download from VMWareSRM site
Note
bull HyperFlex DR Powershell RunbooksmdashNew Powershell cmdlets are included for automated runbookgeneration when using HyperFlex native disaster recovery The New-HXRunbook cmdlet supports thefollowing workflows Test Recovery Planned Migration and Disaster Recovery These runbooks canbe used to orchestrate DR workflows without the requirement for any third-party software
bull Windows Server 2019 with Hyper-VmdashSupport has been added in this release for the Windows Server2019 operating system for Hyper-V based HyperFlex deployments
bull Kubernetes CSI PluginmdashThis release adds support for the HyperFlex CSI (HX-CSI) plugin based onthe Kubernetes Container Storage Interface (CSI) specification Customers can now use the HX-CSIplugin to provision andmanage persistent volumes in Kubernetes version 113 and later Note Kubernetes113 support for Cisco Container Platform amp Openshift Container Platform is forthcoming in theirrespective future releases
bull C480 ML Compute only nodemdashThis release introduces support for C480 ML as a new compute-onlynode for Deep LearningMachine Learning Workloads Data scientists can now use the power of up toeight NVidia SXM2 V100 GPUs to accelerate deep learning workloads VMs running deep-learningworkloads will need to use PCIe pass-through for access to GPUs
bull Higher Capacity DrivesmdashAnew 24TB 10k rpm SAS HDD option for SFF Hybrid HyperFlex clustersand a 12TB 72K rpmSASHDDoption for LFFHybrid are now available Both HyperFlex andHyperFlexEdge support the 24TB capacity point for maximum density in this form factor Note that HyperFlex
Release Notes for Cisco HX Data Platform Release 408
New Features
Edge does not support LFF-drives HyperFlex HyperV version does not yet support the new 12TB driveoption See the HyperFlex spec sheets for a full list of configurable options
bull New Cache and increased scale for Hyper-V mdashNVMe amp Optane SSDs are now supported as cachedrives for Hyper-V deployments Furthermore scale limits have been increased to 16+16(Converged+Compute-only) for both SFF (AF and Hybrid) amp LFF (Hybrid) clusters
bull Centralized Audit Log ExportmdashThis release adds support for audit logging via a remote syslog serverThis capability enables customers to retain audit logs from all HyperFlex nodes in a centralized remotesyslog server to meet retention and compliance requirements
bull DISA STIG CompliancemdashThis release adds new HX REST APIs for setting removing and checkingstatus of DISA STIGs for Controller VMs ESXi hosts and vCenter These APIs enable customers tomeet DISA security requirements by centrally and securely applying STIGs detecting and correctingfor drifts in any STIG settings
Supported Versions and System RequirementsCisco HX Data Platform requires specific software and hardware versions and networking settings forsuccessful installation
For a complete list of requirements see
bull Cisco HyperFlex Systems Installation Guide for VMware ESXi or
bull Cisco HyperFlex Systems Installation Guide for Microsoft Hyper-V
Hardware and Software Interoperability
For a complete list of hardware and software inter-dependencies refer to respective Cisco UCS Managerrelease version of Hardware and Software Interoperability for Cisco HyperFlex HX-Series
HyperFlex Software Versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinan HX Storage Cluster are compatible
bull HyperFlex does not support UCS Manager and UCS Server Firmware versions 40(4a) 40(4b)and 40(4c)
Do not upgrade to these versions of firmware
Do not upgrade to these versions of UCS Manager
Important
bull Verify that the preconfigured HX servers have the same version of Cisco UCS server firmware installedIf the Cisco UCS Fabric Interconnects (FI) firmware versions are different see the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40 for steps to align the firmware versions
bull M4 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M4 or HX220c M4) deploymentsverify that Cisco UCS Manager 31(3k) 32(3i) or 40(2b) or higher is installed For moreinformation see Recommended Cisco HyperFlex HX Data Platform Software Releases
Release Notes for Cisco HX Data Platform Release 409
Supported Versions and System Requirements
bull M5 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M5 or HX220c M5) deploymentsverify that the recommended UCS firmware version is installed
For SED-based HyperFlex systems ensure that the A (Infrastructure) B (Bladeserver) and C (Rack server) bundles are at Cisco UCS Manager version 40(2b)or later for all SED M4M5 systems For more details see CSCvh04307 ForSED-based HyperFlex systems also ensure that all clusters are at HyperFlexRelease 35(2b) or later For more information see Field Notice (70234) andCSCvk17250
Important
bull To reinstall an HX server download supported and compatible versions of the software See theCisco HyperFlex Systems Installation Guide for VMware ESXi for the requirements and steps
bull Important For Intersight edge servers running older than 40(1a) CIMC version HUU is the suggestedmechanism to update the firmware
Table 1 HyperFlex Software Versions for M4M5 Servers
M5 Recommended FIServer Firmware
(be sure to review important notes above)
M4 Recommended FIServer Firmware
(be sure to review important notes above)
HyperFlexRelease
40(4h)40(4h)40(2b)
40(4h)40(4h)40(2a)
40(4h)40(4h)40(1b)
40(4e)40(4e)40(1a)
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Cisco HX Data Platform Release 4x based Deployments
Confirm the component firmware on the server meets the minimum versions listed in the following tables
HyperFlex Edge does not support Cisco IMC versions 40(4a) 40(4b) 40(4c) 40(4d) and 40(4e)Important
Table 2 HX220c M4 HXAF220c M4 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
402d0BIOS
40(2h)Cisco Integrated Management Controller(CIMC)
Release Notes for Cisco HX Data Platform Release 4010
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
13000012SAS HBA
43(2b)VIC 1227
0103 (Toshiba)
CS07 (Intel 480)
SSD
CN03 (Seagate)
CK03 (Seagate 18)
CK03 (Seagate 24)
sF3Q (Samsung)
3F3Q (Samsung)
HDD
40(2h)
Download Software
Host Upgrade Utility (HUU) Download Link
Table 3 HX220c M5 HXAF220c M5 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
C220M5404p00224200755BIOS
40(4i)Cisco Integrated Management Controller(CIMC)
09000008SAS HBA
43(3c)VIC 1387
50(3e)VIC 1457
G201CS01 (Intel)
0104 (Toshiba)
0107 (Toshiba)
SSD
2F3Q (Samsung)
3F3Q (Samsung)
NOA4 (Seagate)
CK03 (Seagate 18tb)
CK03 (Seagate 24)
HDD
Release Notes for Cisco HX Data Platform Release 4011
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
40(4k)
Download Software
Host Upgrade Utility (HUU) Download Link
HyperFlex Licensing
As of version 26(1a) HyperFlex supports VMware PAC licensing Existing VMware embedded licenses willcontinue to be supported
As of version 25(1a) HyperFlex uses a smart licensing mechanism to apply your licenses See the CiscoHyperFlex Systems Installation Guide for VMware ESXi for details and steps
VMware vSphere Licensing Requirements
How you purchase your vSphere license determines how your license applies to your HyperFlex system
bull If you purchased your vSphere license with HyperFlex
Each HyperFlex server either has the Enterprise or Enterprise Plus edition preinstalled at the factory
bull HX Nodes have OEM licenses preinstalled If you delete or overwrite thecontent of the boot drives after receiving the HX servers you also delete thefactory-installed licenses
bull OEM license keys is a newVMware vCenter 60 U1b feature Earlier versionsdo not support OEM licenses
bull All factory-installed HX nodes share the same OEM license key WithvSphere OEM keys the Usage count can exceed the Capacity value
bull When you add an HX host to vCenter through the Add Host wizard in theAssign license section select the OEM license
We obfuscate the actual vSphere OEM license key for example0N085-XXXXX-XXXXX-XXXXX-10LHH
bull Standard Essentials Plus and ROBO editions are not available preinstalledon HX servers
Note
bull If you did NOT purchase your vSphere license with HyperFlex
The HX nodes have a vSphere Foundation license preinstalled After initial setup you can apply thelicense to a supported version of vSphere
bull If you purchased a vSphere PAC license
Follow the instructions in your PAC license letter from VMware to add the license to your MY VMwareaccount then follow the instructions to add your HX host to vCenter and assign the PAC license
Release Notes for Cisco HX Data Platform Release 4012
HyperFlex Licensing
HX Data Platform Software Versions for HyperFlex Witness Node
Witness Node VersionHyperFlex Release
10840(2b)
10840(2a)
10440(1b)
10440(1a)
Software Requirements for VMware ESXi
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and VMware vSphere VMware vCenter and VMware ESXi
bull Verify that all HX servers have a compatible version of vSphere preinstalled
bull Verify that the vCenter version is the same or later than the ESXi version
bull Verify that the vCenter and ESXi versions are compatible by consulting the VMware ProductInteroperability Matrix Newer vCenter versions may be used with older ESXi versions so long as bothESXi and vCenter are supported in the table below
bull Verify that you have a vCenter administrator account with root-level privileges and the associatedpassword
For VIC1457 there is no support for ESXi 60Note
The below table applies for all of the following VMware vSphere Editions Enterprise Enterprise PlusStandard Essentials Plus ROBO
VMware vCenter VersionsVMware ESXi VersionsHyperFlex Version
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2b)1
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2a)2
60 U3 65 U3 67 U260 U3 65 U3 67 U2340(1b)
60 U3 65 U2 67 U260 U3 65 U2 67 U2440(1a)
1 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
2 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
3 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
4 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4013
HX Data Platform Software Versions for HyperFlex Witness Node
Software Requirements for Microsoft Hyper-V
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and Microsoft Hyper-V (Hyper-V) components
HyperFlex Software versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinthe HX Storage Cluster are compatible
bull Cisco HyperFlex M5 Converged nodesmdash For Hybrid (Cisco HyperFlex HX240c M5 HX220c M5)and All Flash (Cisco HyperFlex HXAF240cM5 HXAF220cM5) verify that Cisco UCSManager 40(2b)is installed HX 40 (1a) does not support Hyper-V on the All NVMe (HXAF220C-M5SN) nodes Fordetailed information on installation requirements and steps see theCisco HyperFlex Systems InstallationGuide on Microsoft Hyper-V
Table 4 Supported HyperFlex Software versions for M5 Servers on Hyper-V
M5 Recommended Server FirmwareHyperFlex Release
40(4h)40(2b)
40(4g)40(2a)
40(4e)40(1b)
40(4e)40(1a)
Release Notes for Cisco HX Data Platform Release 4014
Software Requirements for Microsoft Hyper-V
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
bull Scaled-Up HyperFlex Edge ClustersmdashThis release adds support for four node HyperFlex Edge clustersenabling a full range of size options for remote and branch offices Size the branch office environmentsto suit current needs with a two three or four node HyperFlex Edge cluster Cisco Intersight providesfull-lifecycle management and 1GE and 10GE networking options are available
bull Cisco Intersight Invisible Cloud WitnessmdashFor two node clusters this feature eliminates the need forwitness VMs the infrastructure to run those VMs and the management overhead to deploy scale andpatch witnessing software The Invisible Cloud Witness is responsible for maintaining cluster HA in theevent of failure scenarios This feature is included at no additional cost and is automatically deployedand managed by Cisco Intersight
bull Cloud-delivered HyperFlex Edge UpgradesmdashPowered by Cisco Intersight this feature adds supportfor multi-site orchestrated remote upgrades of the HyperFlex Data Platform This feature will be enabledwith the next 40 patch release and will allowHyperFlex Edge clusters deployed via Intersight to performorchestrated upgrades across one or many sites in parallel
bull All-NVMe HyperFlexmdashStarting with this release a new high-end performance node powered by allNVMe drives is available for HyperFlex clusters Co-engineered with Intel to support Intel VMD forhot-plug and surprise removal this offering represents an industry first an enterprise-ready and fullyvalidated all-NVMe HCI appliance The all-NVMe offering is available in the 220 form factor (1RU)and is powered by Intel Optane cache drives for maximum performance and highest endurance
bull VMware Site Recovery Manager (SRM) IntegrationmdashThis release brings support for a Cisco developedStorage Replication Adapter (SRA) for SRM The SRA provides the ability to leverage HyperFlex nativeasync replication with the powerful orchestration and runbook capabilities of SRM The SRA includesthe ability to perform test recoveries planned migrations and full disaster recovery
HX SRA is certified by VMware and is available for download from VMWareSRM site
Note
bull HyperFlex DR Powershell RunbooksmdashNew Powershell cmdlets are included for automated runbookgeneration when using HyperFlex native disaster recovery The New-HXRunbook cmdlet supports thefollowing workflows Test Recovery Planned Migration and Disaster Recovery These runbooks canbe used to orchestrate DR workflows without the requirement for any third-party software
bull Windows Server 2019 with Hyper-VmdashSupport has been added in this release for the Windows Server2019 operating system for Hyper-V based HyperFlex deployments
bull Kubernetes CSI PluginmdashThis release adds support for the HyperFlex CSI (HX-CSI) plugin based onthe Kubernetes Container Storage Interface (CSI) specification Customers can now use the HX-CSIplugin to provision andmanage persistent volumes in Kubernetes version 113 and later Note Kubernetes113 support for Cisco Container Platform amp Openshift Container Platform is forthcoming in theirrespective future releases
bull C480 ML Compute only nodemdashThis release introduces support for C480 ML as a new compute-onlynode for Deep LearningMachine Learning Workloads Data scientists can now use the power of up toeight NVidia SXM2 V100 GPUs to accelerate deep learning workloads VMs running deep-learningworkloads will need to use PCIe pass-through for access to GPUs
bull Higher Capacity DrivesmdashAnew 24TB 10k rpm SAS HDD option for SFF Hybrid HyperFlex clustersand a 12TB 72K rpmSASHDDoption for LFFHybrid are now available Both HyperFlex andHyperFlexEdge support the 24TB capacity point for maximum density in this form factor Note that HyperFlex
Release Notes for Cisco HX Data Platform Release 408
New Features
Edge does not support LFF-drives HyperFlex HyperV version does not yet support the new 12TB driveoption See the HyperFlex spec sheets for a full list of configurable options
bull New Cache and increased scale for Hyper-V mdashNVMe amp Optane SSDs are now supported as cachedrives for Hyper-V deployments Furthermore scale limits have been increased to 16+16(Converged+Compute-only) for both SFF (AF and Hybrid) amp LFF (Hybrid) clusters
bull Centralized Audit Log ExportmdashThis release adds support for audit logging via a remote syslog serverThis capability enables customers to retain audit logs from all HyperFlex nodes in a centralized remotesyslog server to meet retention and compliance requirements
bull DISA STIG CompliancemdashThis release adds new HX REST APIs for setting removing and checkingstatus of DISA STIGs for Controller VMs ESXi hosts and vCenter These APIs enable customers tomeet DISA security requirements by centrally and securely applying STIGs detecting and correctingfor drifts in any STIG settings
Supported Versions and System RequirementsCisco HX Data Platform requires specific software and hardware versions and networking settings forsuccessful installation
For a complete list of requirements see
bull Cisco HyperFlex Systems Installation Guide for VMware ESXi or
bull Cisco HyperFlex Systems Installation Guide for Microsoft Hyper-V
Hardware and Software Interoperability
For a complete list of hardware and software inter-dependencies refer to respective Cisco UCS Managerrelease version of Hardware and Software Interoperability for Cisco HyperFlex HX-Series
HyperFlex Software Versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinan HX Storage Cluster are compatible
bull HyperFlex does not support UCS Manager and UCS Server Firmware versions 40(4a) 40(4b)and 40(4c)
Do not upgrade to these versions of firmware
Do not upgrade to these versions of UCS Manager
Important
bull Verify that the preconfigured HX servers have the same version of Cisco UCS server firmware installedIf the Cisco UCS Fabric Interconnects (FI) firmware versions are different see the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40 for steps to align the firmware versions
bull M4 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M4 or HX220c M4) deploymentsverify that Cisco UCS Manager 31(3k) 32(3i) or 40(2b) or higher is installed For moreinformation see Recommended Cisco HyperFlex HX Data Platform Software Releases
Release Notes for Cisco HX Data Platform Release 409
Supported Versions and System Requirements
bull M5 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M5 or HX220c M5) deploymentsverify that the recommended UCS firmware version is installed
For SED-based HyperFlex systems ensure that the A (Infrastructure) B (Bladeserver) and C (Rack server) bundles are at Cisco UCS Manager version 40(2b)or later for all SED M4M5 systems For more details see CSCvh04307 ForSED-based HyperFlex systems also ensure that all clusters are at HyperFlexRelease 35(2b) or later For more information see Field Notice (70234) andCSCvk17250
Important
bull To reinstall an HX server download supported and compatible versions of the software See theCisco HyperFlex Systems Installation Guide for VMware ESXi for the requirements and steps
bull Important For Intersight edge servers running older than 40(1a) CIMC version HUU is the suggestedmechanism to update the firmware
Table 1 HyperFlex Software Versions for M4M5 Servers
M5 Recommended FIServer Firmware
(be sure to review important notes above)
M4 Recommended FIServer Firmware
(be sure to review important notes above)
HyperFlexRelease
40(4h)40(4h)40(2b)
40(4h)40(4h)40(2a)
40(4h)40(4h)40(1b)
40(4e)40(4e)40(1a)
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Cisco HX Data Platform Release 4x based Deployments
Confirm the component firmware on the server meets the minimum versions listed in the following tables
HyperFlex Edge does not support Cisco IMC versions 40(4a) 40(4b) 40(4c) 40(4d) and 40(4e)Important
Table 2 HX220c M4 HXAF220c M4 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
402d0BIOS
40(2h)Cisco Integrated Management Controller(CIMC)
Release Notes for Cisco HX Data Platform Release 4010
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
13000012SAS HBA
43(2b)VIC 1227
0103 (Toshiba)
CS07 (Intel 480)
SSD
CN03 (Seagate)
CK03 (Seagate 18)
CK03 (Seagate 24)
sF3Q (Samsung)
3F3Q (Samsung)
HDD
40(2h)
Download Software
Host Upgrade Utility (HUU) Download Link
Table 3 HX220c M5 HXAF220c M5 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
C220M5404p00224200755BIOS
40(4i)Cisco Integrated Management Controller(CIMC)
09000008SAS HBA
43(3c)VIC 1387
50(3e)VIC 1457
G201CS01 (Intel)
0104 (Toshiba)
0107 (Toshiba)
SSD
2F3Q (Samsung)
3F3Q (Samsung)
NOA4 (Seagate)
CK03 (Seagate 18tb)
CK03 (Seagate 24)
HDD
Release Notes for Cisco HX Data Platform Release 4011
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
40(4k)
Download Software
Host Upgrade Utility (HUU) Download Link
HyperFlex Licensing
As of version 26(1a) HyperFlex supports VMware PAC licensing Existing VMware embedded licenses willcontinue to be supported
As of version 25(1a) HyperFlex uses a smart licensing mechanism to apply your licenses See the CiscoHyperFlex Systems Installation Guide for VMware ESXi for details and steps
VMware vSphere Licensing Requirements
How you purchase your vSphere license determines how your license applies to your HyperFlex system
bull If you purchased your vSphere license with HyperFlex
Each HyperFlex server either has the Enterprise or Enterprise Plus edition preinstalled at the factory
bull HX Nodes have OEM licenses preinstalled If you delete or overwrite thecontent of the boot drives after receiving the HX servers you also delete thefactory-installed licenses
bull OEM license keys is a newVMware vCenter 60 U1b feature Earlier versionsdo not support OEM licenses
bull All factory-installed HX nodes share the same OEM license key WithvSphere OEM keys the Usage count can exceed the Capacity value
bull When you add an HX host to vCenter through the Add Host wizard in theAssign license section select the OEM license
We obfuscate the actual vSphere OEM license key for example0N085-XXXXX-XXXXX-XXXXX-10LHH
bull Standard Essentials Plus and ROBO editions are not available preinstalledon HX servers
Note
bull If you did NOT purchase your vSphere license with HyperFlex
The HX nodes have a vSphere Foundation license preinstalled After initial setup you can apply thelicense to a supported version of vSphere
bull If you purchased a vSphere PAC license
Follow the instructions in your PAC license letter from VMware to add the license to your MY VMwareaccount then follow the instructions to add your HX host to vCenter and assign the PAC license
Release Notes for Cisco HX Data Platform Release 4012
HyperFlex Licensing
HX Data Platform Software Versions for HyperFlex Witness Node
Witness Node VersionHyperFlex Release
10840(2b)
10840(2a)
10440(1b)
10440(1a)
Software Requirements for VMware ESXi
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and VMware vSphere VMware vCenter and VMware ESXi
bull Verify that all HX servers have a compatible version of vSphere preinstalled
bull Verify that the vCenter version is the same or later than the ESXi version
bull Verify that the vCenter and ESXi versions are compatible by consulting the VMware ProductInteroperability Matrix Newer vCenter versions may be used with older ESXi versions so long as bothESXi and vCenter are supported in the table below
bull Verify that you have a vCenter administrator account with root-level privileges and the associatedpassword
For VIC1457 there is no support for ESXi 60Note
The below table applies for all of the following VMware vSphere Editions Enterprise Enterprise PlusStandard Essentials Plus ROBO
VMware vCenter VersionsVMware ESXi VersionsHyperFlex Version
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2b)1
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2a)2
60 U3 65 U3 67 U260 U3 65 U3 67 U2340(1b)
60 U3 65 U2 67 U260 U3 65 U2 67 U2440(1a)
1 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
2 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
3 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
4 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4013
HX Data Platform Software Versions for HyperFlex Witness Node
Software Requirements for Microsoft Hyper-V
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and Microsoft Hyper-V (Hyper-V) components
HyperFlex Software versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinthe HX Storage Cluster are compatible
bull Cisco HyperFlex M5 Converged nodesmdash For Hybrid (Cisco HyperFlex HX240c M5 HX220c M5)and All Flash (Cisco HyperFlex HXAF240cM5 HXAF220cM5) verify that Cisco UCSManager 40(2b)is installed HX 40 (1a) does not support Hyper-V on the All NVMe (HXAF220C-M5SN) nodes Fordetailed information on installation requirements and steps see theCisco HyperFlex Systems InstallationGuide on Microsoft Hyper-V
Table 4 Supported HyperFlex Software versions for M5 Servers on Hyper-V
M5 Recommended Server FirmwareHyperFlex Release
40(4h)40(2b)
40(4g)40(2a)
40(4e)40(1b)
40(4e)40(1a)
Release Notes for Cisco HX Data Platform Release 4014
Software Requirements for Microsoft Hyper-V
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Edge does not support LFF-drives HyperFlex HyperV version does not yet support the new 12TB driveoption See the HyperFlex spec sheets for a full list of configurable options
bull New Cache and increased scale for Hyper-V mdashNVMe amp Optane SSDs are now supported as cachedrives for Hyper-V deployments Furthermore scale limits have been increased to 16+16(Converged+Compute-only) for both SFF (AF and Hybrid) amp LFF (Hybrid) clusters
bull Centralized Audit Log ExportmdashThis release adds support for audit logging via a remote syslog serverThis capability enables customers to retain audit logs from all HyperFlex nodes in a centralized remotesyslog server to meet retention and compliance requirements
bull DISA STIG CompliancemdashThis release adds new HX REST APIs for setting removing and checkingstatus of DISA STIGs for Controller VMs ESXi hosts and vCenter These APIs enable customers tomeet DISA security requirements by centrally and securely applying STIGs detecting and correctingfor drifts in any STIG settings
Supported Versions and System RequirementsCisco HX Data Platform requires specific software and hardware versions and networking settings forsuccessful installation
For a complete list of requirements see
bull Cisco HyperFlex Systems Installation Guide for VMware ESXi or
bull Cisco HyperFlex Systems Installation Guide for Microsoft Hyper-V
Hardware and Software Interoperability
For a complete list of hardware and software inter-dependencies refer to respective Cisco UCS Managerrelease version of Hardware and Software Interoperability for Cisco HyperFlex HX-Series
HyperFlex Software Versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinan HX Storage Cluster are compatible
bull HyperFlex does not support UCS Manager and UCS Server Firmware versions 40(4a) 40(4b)and 40(4c)
Do not upgrade to these versions of firmware
Do not upgrade to these versions of UCS Manager
Important
bull Verify that the preconfigured HX servers have the same version of Cisco UCS server firmware installedIf the Cisco UCS Fabric Interconnects (FI) firmware versions are different see the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40 for steps to align the firmware versions
bull M4 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M4 or HX220c M4) deploymentsverify that Cisco UCS Manager 31(3k) 32(3i) or 40(2b) or higher is installed For moreinformation see Recommended Cisco HyperFlex HX Data Platform Software Releases
Release Notes for Cisco HX Data Platform Release 409
Supported Versions and System Requirements
bull M5 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M5 or HX220c M5) deploymentsverify that the recommended UCS firmware version is installed
For SED-based HyperFlex systems ensure that the A (Infrastructure) B (Bladeserver) and C (Rack server) bundles are at Cisco UCS Manager version 40(2b)or later for all SED M4M5 systems For more details see CSCvh04307 ForSED-based HyperFlex systems also ensure that all clusters are at HyperFlexRelease 35(2b) or later For more information see Field Notice (70234) andCSCvk17250
Important
bull To reinstall an HX server download supported and compatible versions of the software See theCisco HyperFlex Systems Installation Guide for VMware ESXi for the requirements and steps
bull Important For Intersight edge servers running older than 40(1a) CIMC version HUU is the suggestedmechanism to update the firmware
Table 1 HyperFlex Software Versions for M4M5 Servers
M5 Recommended FIServer Firmware
(be sure to review important notes above)
M4 Recommended FIServer Firmware
(be sure to review important notes above)
HyperFlexRelease
40(4h)40(4h)40(2b)
40(4h)40(4h)40(2a)
40(4h)40(4h)40(1b)
40(4e)40(4e)40(1a)
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Cisco HX Data Platform Release 4x based Deployments
Confirm the component firmware on the server meets the minimum versions listed in the following tables
HyperFlex Edge does not support Cisco IMC versions 40(4a) 40(4b) 40(4c) 40(4d) and 40(4e)Important
Table 2 HX220c M4 HXAF220c M4 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
402d0BIOS
40(2h)Cisco Integrated Management Controller(CIMC)
Release Notes for Cisco HX Data Platform Release 4010
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
13000012SAS HBA
43(2b)VIC 1227
0103 (Toshiba)
CS07 (Intel 480)
SSD
CN03 (Seagate)
CK03 (Seagate 18)
CK03 (Seagate 24)
sF3Q (Samsung)
3F3Q (Samsung)
HDD
40(2h)
Download Software
Host Upgrade Utility (HUU) Download Link
Table 3 HX220c M5 HXAF220c M5 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
C220M5404p00224200755BIOS
40(4i)Cisco Integrated Management Controller(CIMC)
09000008SAS HBA
43(3c)VIC 1387
50(3e)VIC 1457
G201CS01 (Intel)
0104 (Toshiba)
0107 (Toshiba)
SSD
2F3Q (Samsung)
3F3Q (Samsung)
NOA4 (Seagate)
CK03 (Seagate 18tb)
CK03 (Seagate 24)
HDD
Release Notes for Cisco HX Data Platform Release 4011
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
40(4k)
Download Software
Host Upgrade Utility (HUU) Download Link
HyperFlex Licensing
As of version 26(1a) HyperFlex supports VMware PAC licensing Existing VMware embedded licenses willcontinue to be supported
As of version 25(1a) HyperFlex uses a smart licensing mechanism to apply your licenses See the CiscoHyperFlex Systems Installation Guide for VMware ESXi for details and steps
VMware vSphere Licensing Requirements
How you purchase your vSphere license determines how your license applies to your HyperFlex system
bull If you purchased your vSphere license with HyperFlex
Each HyperFlex server either has the Enterprise or Enterprise Plus edition preinstalled at the factory
bull HX Nodes have OEM licenses preinstalled If you delete or overwrite thecontent of the boot drives after receiving the HX servers you also delete thefactory-installed licenses
bull OEM license keys is a newVMware vCenter 60 U1b feature Earlier versionsdo not support OEM licenses
bull All factory-installed HX nodes share the same OEM license key WithvSphere OEM keys the Usage count can exceed the Capacity value
bull When you add an HX host to vCenter through the Add Host wizard in theAssign license section select the OEM license
We obfuscate the actual vSphere OEM license key for example0N085-XXXXX-XXXXX-XXXXX-10LHH
bull Standard Essentials Plus and ROBO editions are not available preinstalledon HX servers
Note
bull If you did NOT purchase your vSphere license with HyperFlex
The HX nodes have a vSphere Foundation license preinstalled After initial setup you can apply thelicense to a supported version of vSphere
bull If you purchased a vSphere PAC license
Follow the instructions in your PAC license letter from VMware to add the license to your MY VMwareaccount then follow the instructions to add your HX host to vCenter and assign the PAC license
Release Notes for Cisco HX Data Platform Release 4012
HyperFlex Licensing
HX Data Platform Software Versions for HyperFlex Witness Node
Witness Node VersionHyperFlex Release
10840(2b)
10840(2a)
10440(1b)
10440(1a)
Software Requirements for VMware ESXi
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and VMware vSphere VMware vCenter and VMware ESXi
bull Verify that all HX servers have a compatible version of vSphere preinstalled
bull Verify that the vCenter version is the same or later than the ESXi version
bull Verify that the vCenter and ESXi versions are compatible by consulting the VMware ProductInteroperability Matrix Newer vCenter versions may be used with older ESXi versions so long as bothESXi and vCenter are supported in the table below
bull Verify that you have a vCenter administrator account with root-level privileges and the associatedpassword
For VIC1457 there is no support for ESXi 60Note
The below table applies for all of the following VMware vSphere Editions Enterprise Enterprise PlusStandard Essentials Plus ROBO
VMware vCenter VersionsVMware ESXi VersionsHyperFlex Version
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2b)1
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2a)2
60 U3 65 U3 67 U260 U3 65 U3 67 U2340(1b)
60 U3 65 U2 67 U260 U3 65 U2 67 U2440(1a)
1 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
2 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
3 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
4 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4013
HX Data Platform Software Versions for HyperFlex Witness Node
Software Requirements for Microsoft Hyper-V
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and Microsoft Hyper-V (Hyper-V) components
HyperFlex Software versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinthe HX Storage Cluster are compatible
bull Cisco HyperFlex M5 Converged nodesmdash For Hybrid (Cisco HyperFlex HX240c M5 HX220c M5)and All Flash (Cisco HyperFlex HXAF240cM5 HXAF220cM5) verify that Cisco UCSManager 40(2b)is installed HX 40 (1a) does not support Hyper-V on the All NVMe (HXAF220C-M5SN) nodes Fordetailed information on installation requirements and steps see theCisco HyperFlex Systems InstallationGuide on Microsoft Hyper-V
Table 4 Supported HyperFlex Software versions for M5 Servers on Hyper-V
M5 Recommended Server FirmwareHyperFlex Release
40(4h)40(2b)
40(4g)40(2a)
40(4e)40(1b)
40(4e)40(1a)
Release Notes for Cisco HX Data Platform Release 4014
Software Requirements for Microsoft Hyper-V
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
bull M5 For NEW hybrid or All Flash (Cisco HyperFlex HX240c M5 or HX220c M5) deploymentsverify that the recommended UCS firmware version is installed
For SED-based HyperFlex systems ensure that the A (Infrastructure) B (Bladeserver) and C (Rack server) bundles are at Cisco UCS Manager version 40(2b)or later for all SED M4M5 systems For more details see CSCvh04307 ForSED-based HyperFlex systems also ensure that all clusters are at HyperFlexRelease 35(2b) or later For more information see Field Notice (70234) andCSCvk17250
Important
bull To reinstall an HX server download supported and compatible versions of the software See theCisco HyperFlex Systems Installation Guide for VMware ESXi for the requirements and steps
bull Important For Intersight edge servers running older than 40(1a) CIMC version HUU is the suggestedmechanism to update the firmware
Table 1 HyperFlex Software Versions for M4M5 Servers
M5 Recommended FIServer Firmware
(be sure to review important notes above)
M4 Recommended FIServer Firmware
(be sure to review important notes above)
HyperFlexRelease
40(4h)40(4h)40(2b)
40(4h)40(4h)40(2a)
40(4h)40(4h)40(1b)
40(4e)40(4e)40(1a)
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Cisco HX Data Platform Release 4x based Deployments
Confirm the component firmware on the server meets the minimum versions listed in the following tables
HyperFlex Edge does not support Cisco IMC versions 40(4a) 40(4b) 40(4c) 40(4d) and 40(4e)Important
Table 2 HX220c M4 HXAF220c M4 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
402d0BIOS
40(2h)Cisco Integrated Management Controller(CIMC)
Release Notes for Cisco HX Data Platform Release 4010
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
13000012SAS HBA
43(2b)VIC 1227
0103 (Toshiba)
CS07 (Intel 480)
SSD
CN03 (Seagate)
CK03 (Seagate 18)
CK03 (Seagate 24)
sF3Q (Samsung)
3F3Q (Samsung)
HDD
40(2h)
Download Software
Host Upgrade Utility (HUU) Download Link
Table 3 HX220c M5 HXAF220c M5 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
C220M5404p00224200755BIOS
40(4i)Cisco Integrated Management Controller(CIMC)
09000008SAS HBA
43(3c)VIC 1387
50(3e)VIC 1457
G201CS01 (Intel)
0104 (Toshiba)
0107 (Toshiba)
SSD
2F3Q (Samsung)
3F3Q (Samsung)
NOA4 (Seagate)
CK03 (Seagate 18tb)
CK03 (Seagate 24)
HDD
Release Notes for Cisco HX Data Platform Release 4011
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
40(4k)
Download Software
Host Upgrade Utility (HUU) Download Link
HyperFlex Licensing
As of version 26(1a) HyperFlex supports VMware PAC licensing Existing VMware embedded licenses willcontinue to be supported
As of version 25(1a) HyperFlex uses a smart licensing mechanism to apply your licenses See the CiscoHyperFlex Systems Installation Guide for VMware ESXi for details and steps
VMware vSphere Licensing Requirements
How you purchase your vSphere license determines how your license applies to your HyperFlex system
bull If you purchased your vSphere license with HyperFlex
Each HyperFlex server either has the Enterprise or Enterprise Plus edition preinstalled at the factory
bull HX Nodes have OEM licenses preinstalled If you delete or overwrite thecontent of the boot drives after receiving the HX servers you also delete thefactory-installed licenses
bull OEM license keys is a newVMware vCenter 60 U1b feature Earlier versionsdo not support OEM licenses
bull All factory-installed HX nodes share the same OEM license key WithvSphere OEM keys the Usage count can exceed the Capacity value
bull When you add an HX host to vCenter through the Add Host wizard in theAssign license section select the OEM license
We obfuscate the actual vSphere OEM license key for example0N085-XXXXX-XXXXX-XXXXX-10LHH
bull Standard Essentials Plus and ROBO editions are not available preinstalledon HX servers
Note
bull If you did NOT purchase your vSphere license with HyperFlex
The HX nodes have a vSphere Foundation license preinstalled After initial setup you can apply thelicense to a supported version of vSphere
bull If you purchased a vSphere PAC license
Follow the instructions in your PAC license letter from VMware to add the license to your MY VMwareaccount then follow the instructions to add your HX host to vCenter and assign the PAC license
Release Notes for Cisco HX Data Platform Release 4012
HyperFlex Licensing
HX Data Platform Software Versions for HyperFlex Witness Node
Witness Node VersionHyperFlex Release
10840(2b)
10840(2a)
10440(1b)
10440(1a)
Software Requirements for VMware ESXi
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and VMware vSphere VMware vCenter and VMware ESXi
bull Verify that all HX servers have a compatible version of vSphere preinstalled
bull Verify that the vCenter version is the same or later than the ESXi version
bull Verify that the vCenter and ESXi versions are compatible by consulting the VMware ProductInteroperability Matrix Newer vCenter versions may be used with older ESXi versions so long as bothESXi and vCenter are supported in the table below
bull Verify that you have a vCenter administrator account with root-level privileges and the associatedpassword
For VIC1457 there is no support for ESXi 60Note
The below table applies for all of the following VMware vSphere Editions Enterprise Enterprise PlusStandard Essentials Plus ROBO
VMware vCenter VersionsVMware ESXi VersionsHyperFlex Version
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2b)1
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2a)2
60 U3 65 U3 67 U260 U3 65 U3 67 U2340(1b)
60 U3 65 U2 67 U260 U3 65 U2 67 U2440(1a)
1 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
2 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
3 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
4 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4013
HX Data Platform Software Versions for HyperFlex Witness Node
Software Requirements for Microsoft Hyper-V
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and Microsoft Hyper-V (Hyper-V) components
HyperFlex Software versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinthe HX Storage Cluster are compatible
bull Cisco HyperFlex M5 Converged nodesmdash For Hybrid (Cisco HyperFlex HX240c M5 HX220c M5)and All Flash (Cisco HyperFlex HXAF240cM5 HXAF220cM5) verify that Cisco UCSManager 40(2b)is installed HX 40 (1a) does not support Hyper-V on the All NVMe (HXAF220C-M5SN) nodes Fordetailed information on installation requirements and steps see theCisco HyperFlex Systems InstallationGuide on Microsoft Hyper-V
Table 4 Supported HyperFlex Software versions for M5 Servers on Hyper-V
M5 Recommended Server FirmwareHyperFlex Release
40(4h)40(2b)
40(4g)40(2a)
40(4e)40(1b)
40(4e)40(1a)
Release Notes for Cisco HX Data Platform Release 4014
Software Requirements for Microsoft Hyper-V
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Recommended Firmware Version - HXDP 4x
(be sure to review important note(s) above)
Component
13000012SAS HBA
43(2b)VIC 1227
0103 (Toshiba)
CS07 (Intel 480)
SSD
CN03 (Seagate)
CK03 (Seagate 18)
CK03 (Seagate 24)
sF3Q (Samsung)
3F3Q (Samsung)
HDD
40(2h)
Download Software
Host Upgrade Utility (HUU) Download Link
Table 3 HX220c M5 HXAF220c M5 Cluster
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
C220M5404p00224200755BIOS
40(4i)Cisco Integrated Management Controller(CIMC)
09000008SAS HBA
43(3c)VIC 1387
50(3e)VIC 1457
G201CS01 (Intel)
0104 (Toshiba)
0107 (Toshiba)
SSD
2F3Q (Samsung)
3F3Q (Samsung)
NOA4 (Seagate)
CK03 (Seagate 18tb)
CK03 (Seagate 24)
HDD
Release Notes for Cisco HX Data Platform Release 4011
HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
40(4k)
Download Software
Host Upgrade Utility (HUU) Download Link
HyperFlex Licensing
As of version 26(1a) HyperFlex supports VMware PAC licensing Existing VMware embedded licenses willcontinue to be supported
As of version 25(1a) HyperFlex uses a smart licensing mechanism to apply your licenses See the CiscoHyperFlex Systems Installation Guide for VMware ESXi for details and steps
VMware vSphere Licensing Requirements
How you purchase your vSphere license determines how your license applies to your HyperFlex system
bull If you purchased your vSphere license with HyperFlex
Each HyperFlex server either has the Enterprise or Enterprise Plus edition preinstalled at the factory
bull HX Nodes have OEM licenses preinstalled If you delete or overwrite thecontent of the boot drives after receiving the HX servers you also delete thefactory-installed licenses
bull OEM license keys is a newVMware vCenter 60 U1b feature Earlier versionsdo not support OEM licenses
bull All factory-installed HX nodes share the same OEM license key WithvSphere OEM keys the Usage count can exceed the Capacity value
bull When you add an HX host to vCenter through the Add Host wizard in theAssign license section select the OEM license
We obfuscate the actual vSphere OEM license key for example0N085-XXXXX-XXXXX-XXXXX-10LHH
bull Standard Essentials Plus and ROBO editions are not available preinstalledon HX servers
Note
bull If you did NOT purchase your vSphere license with HyperFlex
The HX nodes have a vSphere Foundation license preinstalled After initial setup you can apply thelicense to a supported version of vSphere
bull If you purchased a vSphere PAC license
Follow the instructions in your PAC license letter from VMware to add the license to your MY VMwareaccount then follow the instructions to add your HX host to vCenter and assign the PAC license
Release Notes for Cisco HX Data Platform Release 4012
HyperFlex Licensing
HX Data Platform Software Versions for HyperFlex Witness Node
Witness Node VersionHyperFlex Release
10840(2b)
10840(2a)
10440(1b)
10440(1a)
Software Requirements for VMware ESXi
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and VMware vSphere VMware vCenter and VMware ESXi
bull Verify that all HX servers have a compatible version of vSphere preinstalled
bull Verify that the vCenter version is the same or later than the ESXi version
bull Verify that the vCenter and ESXi versions are compatible by consulting the VMware ProductInteroperability Matrix Newer vCenter versions may be used with older ESXi versions so long as bothESXi and vCenter are supported in the table below
bull Verify that you have a vCenter administrator account with root-level privileges and the associatedpassword
For VIC1457 there is no support for ESXi 60Note
The below table applies for all of the following VMware vSphere Editions Enterprise Enterprise PlusStandard Essentials Plus ROBO
VMware vCenter VersionsVMware ESXi VersionsHyperFlex Version
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2b)1
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2a)2
60 U3 65 U3 67 U260 U3 65 U3 67 U2340(1b)
60 U3 65 U2 67 U260 U3 65 U2 67 U2440(1a)
1 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
2 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
3 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
4 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4013
HX Data Platform Software Versions for HyperFlex Witness Node
Software Requirements for Microsoft Hyper-V
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and Microsoft Hyper-V (Hyper-V) components
HyperFlex Software versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinthe HX Storage Cluster are compatible
bull Cisco HyperFlex M5 Converged nodesmdash For Hybrid (Cisco HyperFlex HX240c M5 HX220c M5)and All Flash (Cisco HyperFlex HXAF240cM5 HXAF220cM5) verify that Cisco UCSManager 40(2b)is installed HX 40 (1a) does not support Hyper-V on the All NVMe (HXAF220C-M5SN) nodes Fordetailed information on installation requirements and steps see theCisco HyperFlex Systems InstallationGuide on Microsoft Hyper-V
Table 4 Supported HyperFlex Software versions for M5 Servers on Hyper-V
M5 Recommended Server FirmwareHyperFlex Release
40(4h)40(2b)
40(4g)40(2a)
40(4e)40(1b)
40(4e)40(1a)
Release Notes for Cisco HX Data Platform Release 4014
Software Requirements for Microsoft Hyper-V
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Recommended Firmware Version - HXDP 4x
(be sure to review important notes above)
Component
40(4k)
Download Software
Host Upgrade Utility (HUU) Download Link
HyperFlex Licensing
As of version 26(1a) HyperFlex supports VMware PAC licensing Existing VMware embedded licenses willcontinue to be supported
As of version 25(1a) HyperFlex uses a smart licensing mechanism to apply your licenses See the CiscoHyperFlex Systems Installation Guide for VMware ESXi for details and steps
VMware vSphere Licensing Requirements
How you purchase your vSphere license determines how your license applies to your HyperFlex system
bull If you purchased your vSphere license with HyperFlex
Each HyperFlex server either has the Enterprise or Enterprise Plus edition preinstalled at the factory
bull HX Nodes have OEM licenses preinstalled If you delete or overwrite thecontent of the boot drives after receiving the HX servers you also delete thefactory-installed licenses
bull OEM license keys is a newVMware vCenter 60 U1b feature Earlier versionsdo not support OEM licenses
bull All factory-installed HX nodes share the same OEM license key WithvSphere OEM keys the Usage count can exceed the Capacity value
bull When you add an HX host to vCenter through the Add Host wizard in theAssign license section select the OEM license
We obfuscate the actual vSphere OEM license key for example0N085-XXXXX-XXXXX-XXXXX-10LHH
bull Standard Essentials Plus and ROBO editions are not available preinstalledon HX servers
Note
bull If you did NOT purchase your vSphere license with HyperFlex
The HX nodes have a vSphere Foundation license preinstalled After initial setup you can apply thelicense to a supported version of vSphere
bull If you purchased a vSphere PAC license
Follow the instructions in your PAC license letter from VMware to add the license to your MY VMwareaccount then follow the instructions to add your HX host to vCenter and assign the PAC license
Release Notes for Cisco HX Data Platform Release 4012
HyperFlex Licensing
HX Data Platform Software Versions for HyperFlex Witness Node
Witness Node VersionHyperFlex Release
10840(2b)
10840(2a)
10440(1b)
10440(1a)
Software Requirements for VMware ESXi
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and VMware vSphere VMware vCenter and VMware ESXi
bull Verify that all HX servers have a compatible version of vSphere preinstalled
bull Verify that the vCenter version is the same or later than the ESXi version
bull Verify that the vCenter and ESXi versions are compatible by consulting the VMware ProductInteroperability Matrix Newer vCenter versions may be used with older ESXi versions so long as bothESXi and vCenter are supported in the table below
bull Verify that you have a vCenter administrator account with root-level privileges and the associatedpassword
For VIC1457 there is no support for ESXi 60Note
The below table applies for all of the following VMware vSphere Editions Enterprise Enterprise PlusStandard Essentials Plus ROBO
VMware vCenter VersionsVMware ESXi VersionsHyperFlex Version
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2b)1
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2a)2
60 U3 65 U3 67 U260 U3 65 U3 67 U2340(1b)
60 U3 65 U2 67 U260 U3 65 U2 67 U2440(1a)
1 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
2 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
3 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
4 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4013
HX Data Platform Software Versions for HyperFlex Witness Node
Software Requirements for Microsoft Hyper-V
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and Microsoft Hyper-V (Hyper-V) components
HyperFlex Software versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinthe HX Storage Cluster are compatible
bull Cisco HyperFlex M5 Converged nodesmdash For Hybrid (Cisco HyperFlex HX240c M5 HX220c M5)and All Flash (Cisco HyperFlex HXAF240cM5 HXAF220cM5) verify that Cisco UCSManager 40(2b)is installed HX 40 (1a) does not support Hyper-V on the All NVMe (HXAF220C-M5SN) nodes Fordetailed information on installation requirements and steps see theCisco HyperFlex Systems InstallationGuide on Microsoft Hyper-V
Table 4 Supported HyperFlex Software versions for M5 Servers on Hyper-V
M5 Recommended Server FirmwareHyperFlex Release
40(4h)40(2b)
40(4g)40(2a)
40(4e)40(1b)
40(4e)40(1a)
Release Notes for Cisco HX Data Platform Release 4014
Software Requirements for Microsoft Hyper-V
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
HX Data Platform Software Versions for HyperFlex Witness Node
Witness Node VersionHyperFlex Release
10840(2b)
10840(2a)
10440(1b)
10440(1a)
Software Requirements for VMware ESXi
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and VMware vSphere VMware vCenter and VMware ESXi
bull Verify that all HX servers have a compatible version of vSphere preinstalled
bull Verify that the vCenter version is the same or later than the ESXi version
bull Verify that the vCenter and ESXi versions are compatible by consulting the VMware ProductInteroperability Matrix Newer vCenter versions may be used with older ESXi versions so long as bothESXi and vCenter are supported in the table below
bull Verify that you have a vCenter administrator account with root-level privileges and the associatedpassword
For VIC1457 there is no support for ESXi 60Note
The below table applies for all of the following VMware vSphere Editions Enterprise Enterprise PlusStandard Essentials Plus ROBO
VMware vCenter VersionsVMware ESXi VersionsHyperFlex Version
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2b)1
60 U3 65 U3 67 U360 U3 65 U3 67 U340(2a)2
60 U3 65 U3 67 U260 U3 65 U3 67 U2340(1b)
60 U3 65 U2 67 U260 U3 65 U2 67 U2440(1a)
1 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
2 Cisco HyperFlex Release 40(2) is the last major HyperFlex release that will support vSphere 60 (ESXiand vCenter) due to those versions reaching end of VMware general support on March 12 2020
3 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
4 Use of 67 U2 is not recommended see software advisory for CSCvq06952 and CSCvp88515 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4013
HX Data Platform Software Versions for HyperFlex Witness Node
Software Requirements for Microsoft Hyper-V
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and Microsoft Hyper-V (Hyper-V) components
HyperFlex Software versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinthe HX Storage Cluster are compatible
bull Cisco HyperFlex M5 Converged nodesmdash For Hybrid (Cisco HyperFlex HX240c M5 HX220c M5)and All Flash (Cisco HyperFlex HXAF240cM5 HXAF220cM5) verify that Cisco UCSManager 40(2b)is installed HX 40 (1a) does not support Hyper-V on the All NVMe (HXAF220C-M5SN) nodes Fordetailed information on installation requirements and steps see theCisco HyperFlex Systems InstallationGuide on Microsoft Hyper-V
Table 4 Supported HyperFlex Software versions for M5 Servers on Hyper-V
M5 Recommended Server FirmwareHyperFlex Release
40(4h)40(2b)
40(4g)40(2a)
40(4e)40(1b)
40(4e)40(1a)
Release Notes for Cisco HX Data Platform Release 4014
Software Requirements for Microsoft Hyper-V
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Software Requirements for Microsoft Hyper-V
The software requirements include verification that you are using compatible versions of Cisco HyperFlexSystems (HX) components and Microsoft Hyper-V (Hyper-V) components
HyperFlex Software versions
The HX componentsmdashCisco HX Data Platform Installer Cisco HX Data Platform and Cisco UCSfirmwaremdashare installed on different servers Verify that each component on each server used with and withinthe HX Storage Cluster are compatible
bull Cisco HyperFlex M5 Converged nodesmdash For Hybrid (Cisco HyperFlex HX240c M5 HX220c M5)and All Flash (Cisco HyperFlex HXAF240cM5 HXAF220cM5) verify that Cisco UCSManager 40(2b)is installed HX 40 (1a) does not support Hyper-V on the All NVMe (HXAF220C-M5SN) nodes Fordetailed information on installation requirements and steps see theCisco HyperFlex Systems InstallationGuide on Microsoft Hyper-V
Table 4 Supported HyperFlex Software versions for M5 Servers on Hyper-V
M5 Recommended Server FirmwareHyperFlex Release
40(4h)40(2b)
40(4g)40(2a)
40(4e)40(1b)
40(4e)40(1a)
Release Notes for Cisco HX Data Platform Release 4014
Software Requirements for Microsoft Hyper-V
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Table 5 Supported Microsoft Software versions
VersionMicrosoft Component
Windows Server 2016 Datacenter Core amp DesktopExperience
ForWindows Server 2016Datacenter Coreand Desktop Experience the Windows2016 ISO image should be Update BuildRevision (UBR) 1884 at a minimum
Note
Windows Server 2019Datacenter-DesktopExperienceis supported starting from HXDP 401(a) onwards
For Windows Server 2019 DesktopExperience theWindows 2019 ISO imageshould be Update Build Revision (UBR)107 at a minimum
Note
Windows Server 2019 DatacenterndashCore is notsupported currently
Also note that the following are currently notsupported
OEM activated ISOs and Retail ISOs are notsupported
Earlier versions ofWindows Server such asWindows2012r2 are not supported
Non-English versions of the ISO are not supported
Windows Operating System (Windows OS)
A Windows 2012 or later domain and forestfunctionality level
Active Directory
Supported Microsoft License Editions
The Microsoft Windows Server version that is installed on one or more HyperFlex hosts must be licensed asper Microsoft licensing requirements listed on Microsoft Licensing
Browser Recommendations
Use one of the following browsers to run the listed HyperFlex components These browsers have been testedand approved Other browsers might work but full functionality has not been tested and confirmed
Table 6 Supported Browsers
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
11 or higher11 or higher11 or higherNAMicrosoft InternetExplorer
70 or higher70 or higher57 or higher62 or higherGoogle Chrome
Release Notes for Cisco HX Data Platform Release 4015
Browser Recommendations
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
HX ConnectHX Data PlatformInstaller
Cisco UCS ManagerCisco IntersightBrowser
60 or higher60 or higher45 or higher57 or higherMozilla Firefox
NANA9 or higher10 or higherApple Safari
NANA35 or higherNAOpera
Notes
bull Cisco HyperFlex Connect
The minimum recommended resolution is 1024 X 768
bull Cisco HX Data Platform Plug-in
The Cisco HX Data Platform Plug-in runs in vSphere For VMware Host Client System browserrequirements see the VMware documentation at httpswwwvmwarecomsupportpubs
The HX Data Platform Plug-in is not displayed in the vCenter HTML client You must use the vCenterflash client
bull Cisco UCS Manager
The browser must support the following
bull Java Runtime Environment 16 or later
bull Adobe Flash Player 10 or higher is required for some features
For the latest browser information about Cisco UCSManager refer to themost recent Cisco UCSManagerGetting Started Guide
Cisco HX Data Platform Storage Cluster SpecificationsCluster Limits
bull Cisco HX Data Platform supports up to 100 clusters managed per vCenter as per VMware configurationmaximums
bull Cisco HX Data Platform supports any number of clusters on a single FI domain Each HX convergednode must be directly connected to a dedicated FI port on fabric A and fabric B without the use of a FEXC-series compute only nodes must also directly connect to both FIs B-series compute only nodes willconnect through a chassis IO module to both fabrics In the end the number of physical ports on the FIwill dictate the maximum cluster size and maximum number of individual clusters supported in a UCSdomain
The following table provides Cisco HX Data Platform Storage Cluster Specifications
Release Notes for Cisco HX Data Platform Release 4016
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
AllNVMe -HXAF220c-M5SN
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HXAF220c-M5SN
AllNVMe -HXAF220c-M5SN
ESXi 65or 67only NotsupportedwithESXi 60orHyper-V
HX220cM5 Edge
HXAF220cM5 Edge
HX220cM4 Edge
HXAF220cM4 Edge
HX240c-M5LHX220cM5
HX220cAF M5
HX240cM5
HX240cAF M5
HX220cM4
HX220cAF M4
HX240cM4
HX240cAF M4
HXServers
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
C220M5C240M5B200M4B200M5
C240M5C220M5B200M4B200M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
mdashB200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
B200M5M4M3B260M4B420M4B460M4B480M5C220M5M4M3C240M5M4M3C460M4C480M5
Compute-OnlyUCSB-SeriesC-SeriesServers
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
Convergednodesonly
ConvergedandCompute-onlynodes
ConvergedandCompute-onlynodes
SupportedNodes
Release Notes for Cisco HX Data Platform Release 4017
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
NA(requiresEnterpriseHXDP-PLicense)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
NA(requiresEnterpriseHXDP-PLicense)
M4Convergednodes 3
M5Convergednodes23or 4
RequiresHXDP-ELicense
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-16(12TBdrive islimited toamaximumof 8nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
HXDP-SLicensedNodeLimits
11 ratioofHXDP-StoComputeonlynodes
(MinmdashMax)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes2-8 perSite
Compute-onlynodes0-16 perSite
(up tomaxclustersize)
Convergednodes2-16 perSite
Compute-onlynodes0-21 perSite
(up tomaxclustersize)
Convergednodes3-16(12TBHDDoption isnotsupportedforHyperV)
Compute-onlynodes0-16
Convergednodes3-16
Compute-onlynodes0-16
Convergednodes3-32
ComputeOnlynodes0-32
(up tomaxclustersize)
Convergednodes 3
(requiresHXDP-ELicense)
Convergednodes3-16(12TBdrive islimited toamaximumof 8nodes)
Compute-onlynodes0-32(12TBdrive islimited toamaximumof 16nodes)
Convergednodes3-32(76TBdriveconfigsarelimited tomaximumof 12drivesnodeon AF240)
Compute-onlynodes0-32
(up tomaxclustersize)
HXDP-PLicensedNodeLimits
12 ratioofHXDP-PtoComputeonlynodes
(MinmdashMax)
24 perSite 64percluster
24 perSite 48percluster
32 perSite 64percluster
32326434864MaxClusterSize
Release Notes for Cisco HX Data Platform Release 4018
Cisco HX Data Platform Storage Cluster Specifications
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Stretched Cluster (Availableon ESX Only)
Microsoft Hyper-VVMware ESXiNode
212121111121mdash2121MaxComputetoConvergedratio
NoExpansion
Requires Enterprise license
Requires uniform expansion across both sites
Guidelines and Limitationsbull Starting with release 40(2a) SCVM is no longer needed on a Compute node
bull HX REST API Access Token Management ndash Applications leveraging HX REST APIs should re-useaccess tokens when making API calls Once obtained using the AAA Obtain Access Token API accesstokens are valid for 18 days (1555200 seconds) In addition AAA enforces rate limiting on ObtainAccess Token API requests in a 15 minute window auth can be invoked (successfully) a maximum of5 times A user is allowed to create a maximum of 8 unrevoked tokens Subsequent call to auth willautomatically revoke the oldest issued token tomake room for the new token Amaximum of 16 unrevokedtokens can be present in system In order to prevent brute-force attacks after 10 consecutive failedauthentication attempts a user account is locked for a period of 120 secondsFor more information seeCisco HyperFlex Systems REST API Reference guide
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies toHxConnect also
Upgrade Guidelines
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system
bull Unsupported HX Data Platform 17x 18x 20 21x 25x and 26x clustersmdashUsers from anyversion prior to 26(1a) must step through an intermediate version before upgrading to 40 or later releasesIf you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software releasethat is past the last date of support to the latest suggested release on the Cisco Software Download sitesee Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HXReleases For more informationsee the Software Advisory for CSCvq66867 WARNING Only Use HXDP 26(1e) Upgrade PackageWhen Upgrading From HXDP 18(1a)-18(1e)
bull Hypercheck Health Check UtilitymdashCisco recommends running this proactive health check utility onyour HyperFlex cluster prior to upgrade These checks provide early visibility into any areas that mayneed attention andwill help ensure a seamless upgrade experience For more information see the HyperFlexHealth amp Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck
bull vSphere 67 Software AdvisorymdashDo not upgrade to Cisco HX Data Platform Release 40(1a) whenrunning ESXi 67U1 EP06 (build 11675023) Do not upgrade to 67U1 EP06 (build 11675023) ifrunning Cisco HX Data Platform Release 40(1a) See the Software Advisory CSCvo56350 for furtherdetails
Release Notes for Cisco HX Data Platform Release 4019
Guidelines and Limitations
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
The software build version posted at release will override any other local versions
bull Required vCenter upgrademdashFor enhanced security Cisco HX Data Platform Release 35(1a) or laterrequires the use of TLS 12 Therefore vCenter must be upgraded to 60 U3f or later before upgradingto Cisco HX Data Platform Release 35 or later In addition ESXi should be upgraded as required tomeet HX Data Platform compatibility requirements
bull Minimum HXDP version for upgrademdashHXData Platform clusters running 26(1a) or later may upgradedirectly to 40 using the HX Connect UI
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for upgrade from a pre-35 release
bull Cluster ReadinessmdashEnsure that the cluster is properly bootstrapped and the updated plug-in is loadedbefore proceeding Manual cluster bootstrap is required for HX releases earlier than 35(1a) For moreinformation see theManual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guidefor VMware ESXi Release 40 Do not skip this cluster bootstrap step it is required for all upgradesuntil HX Release 35(1a) Auto bootstrap is supported beginning with HX release 35(1a) For moreinformation see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlexSystems Upgrade Guide for VMware ESXi Release 40
Manual bootstrap is not supported on Intersight clusters
bull Initiating Upgrade―Use the HX Connect UI or CLI stcli commands when upgrading from 25(1a)or later releases Use either the CLI stcli commands or the HX Data Platform Plug-in to the vSphereWeb Client when upgrading from a pre-25(1a) release The vCenter plug-in should not be used forupgrades starting with the 25(1a) release The vCenter plug-in should not be used for upgrades startingwith the 25(1a) release
If the current cluster version is at 35(1a) or above you do not need to use the stcli command Directupgrade to 40 is possible
bull Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during theupgrade window If the upgrade fails you should complete the upgrade as soon as possible
bull ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi basedon the running the HXData Platform version (see the section Software Requirements for VMware ESXi)ESXi compatibility is determined by the major version and update release of ESXi It is generally bestto upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized rebootWhen running a split upgrade first upgrade the HX Data Platform then proceed to upgrade ESXi
bull If Upgrading to vSphere 65
bull Certain cluster functions such as native and scheduled snapshotsReadyClones and Enter or Exit HX Maintenance Mode will not operatefrom the time the upgrade is started until the HX Data Platform upgrade iscomplete
bull After upgrading ESXi using the offline zip bundle use the ESX ExitMaintenance Mode option The HX Exit Maintenance Mode option doesnot operate in the vSphere Web Client until the HX Data Platform upgradeis complete
Note
Release Notes for Cisco HX Data Platform Release 4020
Guidelines and Limitations
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
bull vSphere 60 UpgradesmdashUsers on vSphere 60 migrating to 65 upgrade components in the followingorder
1 Upgrade HX Data Platform and UCS firmware
2 Upgrade HX Data Platform and ESXi
3 Upgrade HX Data Platform only first then upgrade ESXi or UCS firmware or both
bull M4 Server Firmware UpgradesmdashUpgrade server firmware to ensure smooth operation and to correctknown issues Specifically newer SAS HBA firmware is available in this release and is recommendedfor long-term stability
bull Users are encouraged to upgrade to 31(3c) C-bundle or later whenever possible
bull Users running C-bundle versions before 31(2f) must upgrade server firmware by performing acombined upgrade of UCS server firmware (C-bundle) to 31(3c) or later and HX Data Platform to25 Do not split the upgrade into two separate operations
bull If the cluster is already on 31(2f) C-bundle or later you may perform an HX Data Platform onlyor combined upgrade as required
bull M5 Server Firmware UpgradesmdashM5 generation servers must run firmware version 32(2d) or later
bull Firmware Downgrades mdashDowngrading UCSM from the HX-installer is not supported
bull M4M5 Mixed DomainsmdashA mixed domain occurs when a new separate M5 cluster is installed underthe same UCS domain that contains existing M4 clusters Under these conditions orchestrated UCSserver firmware upgrade will not operate until Cisco HX Data Platform Release 26 or later is installedon the M4 clusters Therefore it is best practice to first upgrade UCS server firmware to the latest 31(3)or 32(2) patch release before adding a new M5 cluster to the existing UCS domain Additionally any17 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the samedomain
bull Maintenance WindowmdashIf upgrading both HX Data Platform and UCS firmware you can select eithera combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length ofthe maintenance window Cisco UCSManager infrastructure upgrade is only supported using AutoInstalland the direct server firmware upgrade should be performed only through the upgrade orchestrationframework provided by the HX Data Platform Plug-in
bull Unsupported Self-Encrypting Drives (SEDs)mdashIf adding or replacing self-encrypting drives (SEDs)that have been recently qualified in newer versions of HX Data Platform insert the new drives only afterupgrading HX Data Platform to a compatible version All drives must be SED drives mixing SED andnon-SED is not supported
bullbull Enabling External Host AccessmdashWith Cisco HX Data Platform Release 40(1a) port 445 on themanagement network is blocked for enhanced security Note that prior to 40 port 445 port was openenabling external host access If you are upgrading to 40(1a) from a prior release and would like tocontinue external host access you can use a utility to open select hosts For more information aboutenabling external host access see the ConfiguringHyperFlex Share to SCVMM section in the InstallationGuide for Microsoft Hyper-V
Release Notes for Cisco HX Data Platform Release 4021
Guidelines and Limitations
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Mixed Cluster Expansion Guidelinesbull Expanding existing M4 cluster with M5 converged nodes is supported
bull Expanding existing M5 cluster with M4 converged nodes is not supported
bull Expanding existing mixed M4M5 cluster with M4 or M5 converged nodes is supported
bull Adding any supported compute-only nodes is permitted with all M4 M5 and mixed M4M5 clustersusing the HX Data Platform 26 or later Installer Some example combinations are listed here manyother combinations are possibleExample combinationsExpand mixed M4M5 cluster with compute-only B200 C220 C240 M4M5Expand M4 cluster with compute-only B200 M5 C220 M5 C240M5
bull Only expansion workflow is supported to create a mixed cluster Initial cluster creation with mixedM4M5 servers is not supported
bull All M5 servers must match the form factor (220240) type (HybridAF) security capability (Non-SEDonly) amp disk configuration (QTY capacity and non-SED) of the existing M4 servers
bull HX220-M5 will use a maximum of 6 capacity disks (2 disk slots to remain empty) when mixedwith HX220-M4
bull HX Edge SED LFF Hyper-V and Stretched Clusters do not support mixed M4 and M5 clusters
Security FixesThe following security issues are resolved
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-13132 CVE-2019-9924CSCvq6313840(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-11719 CVE-2019-11727CVE-2019-11729
CSCvq7124040(2a)
Release Notes for Cisco HX Data Platform Release 4022
Mixed Cluster Expansion Guidelines
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2019-1125CSCvr0633940(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2015-9383 CVE-2018-14498CVE-2018-20406 CVE-2018-20852CVE-2019-10160 CVE-2019-13117CVE-2019-13118 CVE-2019-14287CVE-2019-14973 CVE-2019-15903CVE-2019-17546 CVE-2019-18197CVE-2019-18218 CVE-2019-5010CVE-2019-5094 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvs0609440(2a)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2017-13168 CVE-2017-18174CVE-2017-18216 CVE-2018-10876CVE-2018-10877 CVE-2018-10878CVE-2018-10879 CVE-2018-10880CVE-2018-10881 CVE-2018-10882CVE-2018-10902 CVE-2018-10938CVE-2018-12233 CVE-2018-12896CVE-2018-13053 CVE-2018-13094CVE-2018-13096 CVE-2018-13405CVE-2018-13406 CVE-2018-14609CVE-2018-14617 CVE-2018-14633CVE-2018-14734 CVE-2018-15572CVE-2018-15594 CVE-2018-16276CVE-2018-16658 CVE-2018-17182CVE-2018-17972 CVE-2018-18021CVE-2018-18690 CVE-2018-18710CVE-2018-6554 CVE-2018-6555CVE-2018-9363
CSCvp6501940(2a)
This vulnerability is due to insufficientHTML iframe protection An attackercould exploit this vulnerability bydirecting a user to an attacker-controlledweb page that contains amaliciousHTMLiframe A successful exploit could allowthe attacker to conduct clickjacking orother clientside browser attacks
NACSCvo9851640(2a)
Release Notes for Cisco HX Data Platform Release 4023
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DescriptionCVEDefect IDRelease
The vulnerability is due to insufficientauthentication for the statistics collectionservice An attacker could exploit thisvulnerability by sending properlyformatted data values to the statisticscollection service of an affected deviceA successful exploit could allow theattacker to cause the web interfacestatistics view to present invalid data tousers
NACSCvj9558440(2a)
The vulnerability is due to insufficientCSRF protections for the web UI on anaffected device An attacker could exploitthis vulnerability by persuading a user ofthe interface to follow a malicious linkA successful exploit could allow theattacker to perform arbitrary actions withthe privilege level of the affected user
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvp2434340(2a)
Vulnerabilities with a version of cJSONidentified by CVE-2019-11834 andCVE-2019-11835
For more information see the relatedCisco Security Advisory
CVE-2019-11834 CVE-2019-11835CSCvq1994940(2a)
HX ESXi image patches for MCEPSCand TAA vulnerabilities with VMwareESXi identified by CVE-2018-12207 andCVE-20190-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439840(2a)
CP networking vulnerabilities affectingthe Linux kernel identified byCVE-2019-11477 CVE-2019-11478CVE-2019-11479
For more information see the relatedCisco Security Advisory
CVE-2019-11477 CVE-2019-11478CVE-2019-11479
CSCvq1954640(2a)
Qualification of Microsoft securitypatches for vulnerabilities withMicrosoftHyper-V hypervisor identified withCVE-2018-12207 CVE-2019-11135
For more information see the relatedCisco Security Advisory
CVE-2018-12207 CVE-2019-11135CSCvr5439940(2a)
Release Notes for Cisco HX Data Platform Release 4024
Security Fixes
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DescriptionCVEDefect IDRelease
OpenSSH vulnerabilities identified byCVE-2019-6111
For more information see the relatedCisco Security Advisory
CVE-2016-10708 CVE-2018-15473CVE-2018-20685 CVE-2019-6109CVE-2019-6111
CSCvp7646340(2a)
Multiple vulnerabilities associated withOpenSSL and LibNSS
For more information see the relatedCisco Security Advisory
CVE-2016-2105 CVE-2016-2106CVE-2016-2107 CVE-2016-2109CVE-2016-2176 CVE-2016-2177CVE-2016-2178 CVE-2016-2179CVE-2016-2180 CVE-2016-2181CVE-2016-2182 CVE-2016-2183CVE-2016-6302 CVE-2016-6303CVE-2016-6304 CVE-2016-6305CVE-2016-6306 CVE-2016-6307CVE-2016-6308 CVE-2016-7055CVE-2016-8610 CVE-2017-3731CVE-2017-3732 CVE-2017-3735CVE-2017-3736 CVE-2017-3737CVE-2017-3738 CVE-2018-0495CVE-2018-0732 CVE-2018-0734CVE-2018-0735 CVE-2018-0737CVE-2018-0739 CVE-2018-12384CVE-2018-12404 CVE-2018-5407CVE-2019-1559
CSCvp6667940(2a)
libpng vulnerability identified byCVE-2016-10087
For more information see the relatedCisco Security Advisory
CVE-2016-10087CSCvp6655540(2a)
Vulnerabilities associated with libjpg andlibpng
For more information see the relatedCisco Security Advisory
CVE-2014-9092 CVE-2016-3616CVE-2017-15232 CVE-2018-11212CVE-2018-11213 CVE-2018-11214CVE-2018-1152 CVE-2018-13785
CSCvp3458640(2a)
Vulnerabilities associated with Glibidentified by CVE-2018-16428 andCVE-2018-16429
For more information see the relatedCisco Security Advisory
CVE-2018-16428 CVE-2018-16429CSCvp3120740(2a)
Multiple vulnerabilities associated withcurl expat python 27 python 35 36and 37 freetype and giflib
For more information see the relatedCisco Security Advisory
CVE-2019-15133 CVE-2019-15903CVE-2019-5010 CVE-2019-5481CVE-2019-5482 CVE-2019-9636CVE-2019-9740 CVE-2019-9947CVE-2019-9948
CSCvr3690340(2a)
Release Notes for Cisco HX Data Platform Release 4025
Security Fixes
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DescriptionCVEDefect IDRelease
Multiple third party softwarevulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2019-14379 CVE-2019-12384CVE-2019-14439
CSCvq9203240(2a)
Vulnerabilities associated with elfutils
For more information see the relatedCisco Security Advisory
CVE-2018-16062 CVE-2018-16402CVE-2018-16403 CVE-2018-18310CVE-2018-18520 CVE-2018-18521CVE-2019-7149 CVE-2019-7150CVE-2019-7665
CSCvq4325040(2a)
Vulnerabilities associated with vim
For more information see the relatedCisco Security Advisory
CVE-2017-5953 CVE-2019-12735CSCvq4323040(2a)
Vulnerabilities associated with sqlite3
For more information see the relatedCisco Security Advisory
CVE-2016-6153 CVE-2017-10989CVE-2017-13685 CVE-2017-2518CVE-2017-2519 CVE-2017-2520CVE-2018-20346 CVE-2018-20505CVE-2018-20506 CVE-2019-8457CVE-2019-9936 CVE-2019-9937
CSCvq4321340(2a)
Vulnerabilities associated with glib20
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320940(2a)
Vulnerabilities associated with expat
For more information see the relatedCisco Security Advisory
CVE-2018-20843CSCvq4320540(2a)
Vulnerabilities associated with bzip2
For more information see the relatedCisco Security Advisory
CVE-2016-3189 CVE-2019-12900CSCvq4319440(2a)
Mulitple Vulnerabilities associated wtihNodeJS
For more information see the relatedCisco Security Advisory
CVE-2018-12115 CVE-2018-0734CVE-2018-5407 CVE-2018-12120CVE-2018-12121 CVE-2018-12122CVE-2018-12123 CVE-2018-12116CVE-2019-5737 CVE-2019-5739
CSCvq1069440(2a)
Vulnerabilities associated with PalletsJinja strformat_map and identified byCVE-2019-10906 and CVE-2016-10745
For more information see the relatedCisco Security Advisory
CVE-2019-10906 CVE-2016-10745CSCvq1038840(2a)
Release Notes for Cisco HX Data Platform Release 4026
Security Fixes
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DescriptionCVEDefect IDRelease
Multiple Security Vulnerabilities
For more information see the relatedCisco Security Advisory
CVE-2014-8501 CVE-2014-9939CVE-2015-9262 CVE-2016-10087CVE-2016-2226 CVE-2016-4487CVE-2016-4488 CVE-2016-4489CVE-2016-4490 CVE-2016-4491CVE-2016-4492 CVE-2016-4493CVE-2016-6131 CVE-2018-10963CVE-2018-13785 CVE-2018-17100CVE-2018-17101 CVE-2018-18557CVE-2018-18661 CVE-2018-7456CVE-2018-8905
CSCvq0917840(2a)
Vulnerabilities associated withlibseccomp
For more information see the relatedCisco Security Advisory
CVE-2019-9893CSCvq0756840(2a)
Vulnerabilities associated withgdk-pixbuf
For more information see the relatedCisco Security Advisory
CVE-2017-12447CSCvq0675540(2a)
Vulnerabilities associated withpython-crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp9381740(2a)
Vulnerabilities associated with wget
For more information see the relatedCisco Security Advisory
CVE-2018-20483 CVE-2019-5953CSCvp8672140(2a)
Multiple vulnerabilities associated withPython Crypto
For more information see the relatedCisco Security Advisory
CVE-2018-6594CSCvp6674840(2a)
Vulnerabilities associated withsensible-utils
For more information see the relatedCisco Security Advisory
CVE-2017-17512CSCvp6673440(2a)
Multiple vulnerabilities associated withPython
For more information see the relatedCisco Security Advisory
CVE-2018-1000030CSCvp6668940(2a)
Release Notes for Cisco HX Data Platform Release 4027
Security Fixes
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DescriptionCVEDefect IDRelease
Multiple patch vulnerabilities identifiedby CVE-2016-10713
For more information see the relatedCisco Security Advisory
CVE-2016-10713CVE-2018-1000156 CVE-2018-6951
CSCvp6667240(2a)
Vulnerabilities associatedwith Little CMSidentified by CVE-2016-10165
For more information see the relatedCisco Security Advisory
CVE-2016-10165 CVE-2018-16435CSCvp6666440(2a)
Vulnerabilities associated with TenableScan identified byCVE-2011-5325CVE-2019-11068CVE-2016-7076CVE-2017-1000368
For more information see the relatedCisco Security Advisory
CVE-2011-5325 CVE-2016-7076CVE-2017-1000368CVE-2019-11068
CSCvp6474640(2a)
Vulnerabilities associated with libxml2294 and earlier as used in XMLSec1223 and earlier and other products andidentified by CVE-2016-9318
For more information see the relatedCisco Security Advisory
CVE-2016-9318 CVE-2017-16932CVE-2017-18258 CVE-2018-14404CVE-2018-14567
CSCvp3479240(2a)
Vulnerabilities associated with LFTPRemote File Names Unauthorized AccessVulnerability and identified byCVE-2018-10916
For more information see the relatedCisco Security Advisory
CVE-2018-10916CSCvp2926640(2a)
Vulnerabilities associated withCryptographyDeprecationWarning signerand verifier have been deprecated
For more information see the relatedCisco Security Advisory
CVE-2018-7750CSCvo3409740(2a)
Multiple Vulnerabilities in python python27 35
For more information see the relatedCisco Security Advisory
CVE-2019-16056CSCvr3979340(2a)
Release Notes for Cisco HX Data Platform Release 4028
Security Fixes
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DescriptionCVEDefect IDRelease
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toexecute commands as the root user
The vulnerability is due to insufficientinput validation An attacker could exploitthis vulnerability by connecting to thecluster service manager and injectingcommands into the bound process Asuccessful exploit could allow the attackerto run commands on the affected host asthe root user
Cisco has released software updates thataddress this vulnerability There areworkarounds that address thisvulnerability
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvq2417640(1b)35(2g)
A vulnerability in the cluster servicemanager of Cisco HyperFlex could allowan unauthenticated adjacent attacker toperform a command injection as the rootuser
The vulnerability is due to an unprotectedlistening interface An attacker couldexploit this vulnerability by connectingto the listening interface and injectingcommands to the bound process Anexploit could allow the attacker to runcommands on the affected host as the rootuser
For more information see the relatedCisco Security Advisory
CVE-2018-15380CSCvj9560640(1b)35(2g)
Release Notes for Cisco HX Data Platform Release 4029
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DescriptionCVEDefect IDRelease
The vulnerabilities associated with theJVM18U121memory leak found duringVC alarms (concurrent 40 calls usingREST API)
CVE-2017-10053 CVE-2017-10067CVE-2017-10074 CVE-2017-10078CVE-2017-10081 CVE-2017-10087CVE-2017-10089 CVE-2017-10090CVE-2017-10096 CVE-2017-10101CVE-2017-10102 CVE-2017-10107CVE-2017-10108 CVE-2017-10109CVE-2017-10110 CVE-2017-10111CVE-2017-10115 CVE-2017-10116CVE-2017-10118 CVE-2017-10135CVE-2017-10176 CVE-2017-10193CVE-2017-10198 CVE-2017-10243CVE-2017-10274 CVE-2017-10281CVE-2017-10285 CVE-2017-10295CVE-2017-10345 CVE-2017-10346CVE-2017-10347 CVE-2017-10348CVE-2017-10349 CVE-2017-10350CVE-2017-10355 CVE-2017-10356CVE-2017-10357 CVE-2017-10388CVE-2017-3509 CVE-2017-3511CVE-2017-3526 CVE-2017-3533CVE-2017-3539 CVE-2017-3544CVE-2018-2579 CVE-2018-2582CVE-2018-2588 CVE-2018-2599CVE-2018-2602 CVE-2018-2603CVE-2018-2618 CVE-2018-2629CVE-2018-2633 CVE-2018-2634CVE-2018-2637 CVE-2018-2641CVE-2018-2663 CVE-2018-2677CVE-2018-2678 CVE-2018-2783CVE-2018-2790 CVE-2018-2794CVE-2018-2795 CVE-2018-2796CVE-2018-2797 CVE-2018-2798CVE-2018-2799 CVE-2018-2800CVE-2018-2814 CVE-2018-2815CVE-2018-2952 CVE-2018-3136CVE-2018-3139 CVE-2018-3149CVE-2018-3150 CVE-2018-3169CVE-2018-3180 CVE-2018-3183CVE-2018-3214 CVE-2019-2422
CSCvo8899740(1b)
Tomcat and Nginx logs are not beingcollected in the support bundle generatedthrough HX Connect in Release 35
NACSCvm5803140(1b)
The vulnerabilities associated with thelibmspack software package versionincluded in Cisco HX Data Platform
CVE-2018-18584
CVE-2018-18585
CSCvn3511940(1a)
Release Notes for Cisco HX Data Platform Release 4030
Security Fixes
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DescriptionCVEDefect IDRelease
The vulnerabilities associated withFasterXML Jackson-Databind TimeValueField Denial of Service
CVE-2018-14719
CVE-2018-14720
CVE-2018-1000873
CVE-2018-14721
CVE-2018-19360
CVE-2018-19362
CVE-2018-19361
CVE-2018-14718
CSCvn8228240(1a)
The vulnerabilities associated with theOpenSSL Protocol software packageversion included in Cisco HX DataPlatform
CVE-2013-3587CSCvo0505440(1a)
The vulnerabilities associated withThird-Party Software Denial of Service
CVE-2018-16487
CVE-2018-19361
CSCvo2781840(1a)
The vulnerabilities associated withLinuxkernel for Ubuntu 1710
CVE-2018-1092
CVE-2018-7492
CVE-2018-8087
CVE-2018-1068
CVE-2018-8781
CSCvm5314935(2a)
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)25(1d)stcli services dns remove should remove the DNS serverinfo from the interface files
CSCvs70967
40(2b)40(1b)HX 40(1b) new deployment from Intersight productioncloud
User receives following error while enabling CSI(Kubernetes integration)
Failure occurred during volume_access_enable Errorwas Nonzero exit code 1
CSCvt10522
40(2b)40(2a)Receive the following alertevent in HX Connect HXController VM HOSTNAME one or more configuredDNS servers not responding
CSCvt13947
Release Notes for Cisco HX Data Platform Release 4031
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)40(2a)API UI CLI show few Drive slots Empty after upgradeto 402a
The drives are seen in the sysmtool lsscsi and stclicommand outputs
The cluster is healthy with no errors
CSCvt14914
40(2b)35(2d)After successfully changing the DNS server on the HXcontroller we still can see the original DNS entry thatwas added during the deployment
CSCvs69154
40(2b)35(2e)HyperFlex Connect and VCenter show APD alarms forHX and non HX datastores This implies there is an issuewith HyperFlex when there may not be
CSCvs30080
40(2b)40(2a)An issue where all the disks in the node get locked afterrebooting the node
We successfully unlocked using sed-clientsh -Ucommand but wanted to test with another reboot anddrives locked again
CSCvs74286
40(2b)40(2a)When running stcli license commands on HyperFlexthe following error is seen
rootSpringpathControllertmp stcli license show all
Show smart licensing failed
Smart Agent is not ready please wait a minute and tryagain
CSCvt13929
40(2b)40(2a)After upgrading to HX 40(2a) the first RoboEdge nodethat gets rebooted then ignores persistent drivesPIDHX-HD24TB10K4KN
Cluster will be in degraded state and have reducedcapacity
CSCvt20144
40(2b)35(2b)Zookeeper fails to start while Exhibitor is runninghowever echo srvr returns nothing
CSCvs21562
40(2b)40(2a)When performing an HyperFlex upgrade a validationwarning may occur due to the host not having EnterprisePlus or Enterprise hypervisor licensing
Upgrade Validation Warning
ESXi host esx1labtest should be configured withVMware Enterprise license for upgrade to continue
CSCvs91787
Release Notes for Cisco HX Data Platform Release 4032
Resolved Caveats in Release 40(2b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2b)35(2g)Cluster expansion fails at Config Installer stage whenthe root and admin password for the storage controller(SCVM) are different
CSCvs69317
40(2b)35(2g)Panic while upgrading ESXiCSCvt06983
40(2b)40(2a)A cluster node running may hang in the Linux kernelThis is classified as an oops and a deviation from theexpected behavior
CSCvs54285
40(2b)35(2g)Rebalance failing on 10+10 node HX 35(2g) stretchcluster
CSCvs69007
40(2b)35(2d)The cluster becomes inaccessible to the IOVisorCSCvr54687
40(2b)40(2a)Panic on the storage controllerCSCvt61297
40(2b)40(2a)The size of support bundle is very large when collectedwith storfs-support command
CSCvt63306
40(2b)40(2a)storfs crash associated withFileSystemUsageWarningEvent orFileSystemUsageAlertEvent due to high usage ofvarstv above 80
CSCvt72807
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
40(2a)35(2d)In normal scenarios HX upgrade gets pendingACK afterupdating service profile In this case UCS only upgradewas stalled because after updating service profile HXupgrade was not getting pending ACK and it was alwayswaiting state
CSCvo39912
40(2a)35(1a)When using motherBoardReplace-12 to clean ZK fromold stNodepNodes resulted in unmounted datastore andmaking the size of the Hyperflex Datastores 0 resultingin all VMs in the cluster going offline
CSCvq39471
40(2a)26(1e)Upgrading cluster and getting error
Failed upgrade validations Checking vCenterconfiguration Reason Upgrade validations failed DRSFault Insufficient resources to satisfy configured failover
CSCvm77294
Release Notes for Cisco HX Data Platform Release 4033
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)The cluster expand fails on a node with DR replicationconfigured
When a HX cluster which has DR replication configuredis expanded we see the installer UI pulling in thereplicationVLAN information instead of themanagementVLAN information Even if we change that informationto the correct mgmt VLAN id and name it it does notseem to work as the node is configured with the VLANof the replicationVLAN in ESXi This leads to the failureof the node add with the host unreachable error
CSCvo70650
40(2a)35(1a)Customer reported that server didnt complete thefirmware upgrade automatically one by one
The user manually needs to put the host into maintenancemode and they manually acknowledged the pendingrequests to finish the UCS firmware upgrade
Maintenance policy is set to default (User-ack) as perthe design
CSCvo91624
40(2a)35(1i)When the cluster is in Failed state and stcli noderemove is attempted the output shows successful thoughthe node failed to get removed from the cluster
CSCvo93017
40(2a)35(1a)HyperFlex cluster upgrade may fail during validationwith a DRS Validation failed error
CSCvp31021
40(2a)35(2b)HX cluster expansion will fail with the following errormessage
MAC address pool configuration failure150[ErrorDescription] bad address block range definitioncollision
CSCvp58318
40(2a)35(2b)Memory usage by carbon cacheCSCvq34873
40(2a)35(2g)Upgrading ESXi cluster fails with error Nodemaintenance mode failed
CSCvr03240
40(2a)35(2e)storfs process does not automatically start on Exit HXMaintenance Mode or other tasks which restart StorageController
CSCvr88978
40(2a)35(2a)Post install script fails with message
Failed to execute ipmitool on HX node
CSCvp10707
40(2a)40(1a)HyperFlex expansionworkflow doesnt pull VLANnamecorrectly
CSCvp46539
Release Notes for Cisco HX Data Platform Release 4034
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)During HX cluster deployment - Validate ClusterCreation phase - HX Installer might fail with the error
fromvarlogspringpathstDeploylog
CSCvq45087
40(2a)35(2b)HX Installer fails to login to the SOL for configuringESX hosts
CSCvq91380
40(2a)35(2d)If DRS is enabled with configuration parameters otherthen fully automated ie manual or partially automatedthan this will change to fully automated during clusterexpansion
CSCvr44222
40(2a)35(1a)One of the validation task name has a typo The taskname is Validating node for mixed of SED and non-SEDdisks
It should be Validating node for mix of SED andnon-SED disks
CSCvp82175
40(2a)30(1i)This combination of using smaller NVMe drives is notsupported in expand operation For example if the drivesare only 375GB and they cannot be added to an existingcluster with larger caching SSD
CSCvo12359
40(2a)35(1a)HyperFlex includes a version of OpenSSL that is affectedby the vulnerabilities identified by the followingCommon Vulnerability and Exposures
CVE-2018-0495
CSCvp66679
40(2a)30(1e)SDK invoke exception when taking a native snapshot ona VM with GPU pci passthrough attached
CSCvo00511
40(2a)30(1i)Node replacement script fails due to EAM errorCSCvo62867
40(2a)35(1a)While pairing a cluster (HX release gt= 35) and cluster(HX release lt 35) remote replication network test failson the cluster (HX release lt 35)
CSCvo79760
40(2a)30(1b)Fixed the issue in the latest support-workflow bundleThis should not be hit anymore
CSCvo87061
40(2a)35(2a)On a 3 node cluster the MBreplace script fails as thescript looks for HEALTHY cluster status
CSCvo87080
40(2a)35(2a)MbReplace script (tar) hangs when running on 352b or40
CSCvp12359
Release Notes for Cisco HX Data Platform Release 4035
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2a)HX replication cleanup failing with error NA 35(2a) -INFODR state is not clean
CSCvp63958
40(2a)40(1a)The check_stig_parameters API incorrectly shows thecompliance state of the cluster as not STIG compliant
CSCvp66277
40(2a)35(2c)The snapshot creation on CBT enabled VM fails witherror Failed in vmreparent vmkfstools clone1
CSCvq06952
40(2a)35(2a)HyperFlex GUI when setting up remote DR-replicationpartnerpeer should show remote replication partner TCPport reachability test results
This will provide quick upfront connectivity results thatfrequently takes customer a while to manually check andconfirm
If reachability tests fail an info window and HX alertshould trigger that list the destination peer IP and thespecific port(s) that could not be reache
CSCvo60587
40(2a)35(2b)If you have more than one IP pool configured for areplication network and local replication network testsare failing you might be running into this issue
CSCvr67130
40(2a)35(2a)Skip task option in MBreplace and Software redeployscripts The scripts should have the option to skip tasksas user inputs and not by modifying the json file
CSCvp05204
40(2a)40(1a)Error noticed on SCVM Console print_req_error IOerror dev fd0 sector 0 on HyperV
It doesnt come up on SSH session
CSCvq34357
40(2a)30(1d)The VMNetwork switch is already created as part of theinstall process
Ideally the ip address assignment should be done duringthe initial install process as well No post install stepsshould be required altrnatively it could be done as partof a post install script
CSCvk36222
40(2a)25(1c)storfs service impacted during HX upgrade from HX21(1) to 25(1c)
HX node which is not being upgraded storfs servicestopped causing outage
CSCvg53223
40(2a)35(1b)HX Stretch Cluster Witness VM reverts to DHCP onreboot
CSCvp37536
40(2a)35(2b)HyperFlex Stretch Cluster Witness does not getprogrammed with an NTP server
CSCvp46578
Release Notes for Cisco HX Data Platform Release 4036
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)35(2b)HyperFlex cluster does not heal after maintenance onnodeserver
CSCvr18528
40(2a)40(2a)High percentage of packet loss on the eth1 data interface9K MTU
CSCvr97089
40(2a)35(2f)The HyperFlex cluster healing state is stuck at 87 andis unable to progress
CSCvr16760
40(2a)40(2a)In some circumstances two replications can occur forthe same VM
From the GUI the replication status for this VM willshow stuck in an In Progress state but there are nojobs stuck in replication layer Subsequent Replicationsshould succeed
CSCvq22898
40(2a)35(2a)After an unexpected power outage the hyperflex clusteris unable to come up correctly
All processes are running time is synced and all nodescan vmkping each other as required
CSCvp13990
40(2a)40(1a)When specifying IP address instead of node ID stclinode maintenanceMode commands fail
CSCvp33657
40(2a)40(1a)2 node edge cluster if experiencing Intersightarbitratorconnectivity issues should NOT allowHXMaintenancemode to initiate as this will bring the cluster down
CSCvr31573
40(2a)35(2c)Old files in varsupportZKTxnlog are not purged withthe daily zklog-cleanup cron job
CSCvr89066
40(2a)35(2a)When a node is in MaintenanceMode any disk removalor replacement will be reflected in UI only after the nodeis brought back frommaintenancemode This is becausestorfs is not running on the node in maintenance andwill not be able to detect disk activities until it is broughtout of Maintenance Mode
CSCvo86431
40(2a)35(2c)Cluster might go offline and stop serving IO requestsrootcvm~ stcli cluster storage-summary--detailGet cluster storage summary failedjavanetConnectException Connection refused19216814210010207
CSCvr01645
40(2a)35(2d)A stretch cluster deployment fails with a messageFormatting nodes Node in Use on the Hyperflex installerand crmZoneType value is seen as 1 in storfscfg oroptspringpathconfigstretchtunes
CSCvq80340
Release Notes for Cisco HX Data Platform Release 4037
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)30(1c)During upgrade we upload the upgrade packagedownloaded from Ciscocom Upgrade packages are offile type tgz hxconnect acceptsstorfs-packages-401a-33028tar file a different type ofcompression and allows upgrade to start and fail whenit cannot decompress the file for bootstrap and scvmclientupgrade on ESXi
CSCvq63888
40(2a)40(2a)While attempting to install hyperflex via intersight thefollowing error is seen
Failed in Task Add hosts to vCenter Cluster with ErrorTry adding hosts manually to vCenter and retry Failedto get addressFailed to host by namelookupHostnamecompanycom on 000053 read udp12700145430-gt12700153 read connection refused
CSCvr52098
40(2a)35(2a)CoreAPI call may time out for cluster management APIwhile doing cluster create validation
CSCvo92952
40(2a)35(2d)Validation fails with Mixed mode expansion checkdue to Enhanced vMotion Compatibility Incompatibilitywhen the ESXi version is different
CSCvq95460
40(2a)30(1i)This failure is seen in installer when custom workflowis used hypervisor configuration + deploy (clean diskpartition=no) at Installing software packages on storagecontroller VM
CSCvr66309
40(2a)35(2d)The HX Installer apparently replaced a comma bysemicolon on the VLAN ID range while exporting theconfiguration to a JSON file
CSCvq93831
40(2a)40(1a)When using unsupported browsers HX Connect userswill not be able to login These users see an errorindicating unauthorized user
CSCvp22693
Release Notes for Cisco HX Data Platform Release 4038
Resolved Caveats in Release 40(2a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(2a)40(1a)A vulnerability in the web-based management interfaceof Cisco HyperFlex Software could allow anunauthenticated remote attacker to conduct a cross-siterequest forgery (CSRF) attack on an affected system
The vulnerability is due to insufficient CSRF protectionsfor the web UI on an affected device An attacker couldexploit this vulnerability by persuading a user of theinterface to follow a malicious link A successful exploitcould allow the attacker to perform arbitrary actions withthe privilege level of the affected user
Cisco has released software updates that address thisvulnerability There are no workarounds that address thisvulnerability
This advisory is available at the following link
httpstoolsciscocomsecuritycentercontentCiscoSecurityAdvisorycisco-sa-20190807-hypflex-csrf
CSCvp24343
40(2a)35(2d)Add message to HX Connect Progress flow to notAcknowledge Pending Activities and reboot the serverson UCSM
HX Connect is doing a controlled rolling server upgradein the background
CSCvq22844
40(2a)40(1b)Compute Node expand operation fails due to incorrectcluster name
CSCvr43786
40(2a)26(1e)RF-2 cluster shutdown during data resynchronizationafter a node failure (non storfs) followed by multipledisk read failures and a HardBlacklist of a disk
CSCvp41241
40(2a)40(1a)HyperFlex Edge nodes do not properly set the ESXihostname during deployment
CSCvo13143
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4039
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4040
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1b) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401b-33133p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4041
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4042
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1b)40(1a)While running the HyperFlex installer with WindowsServer Hyper-V cluster creation process fails with theerror Failure occurred during ClusterCreation process Unable to post thecontent to the downstream This symptomis encountered while deploying HyperFlex cluster withHyperFlex nodes configuredordered with Cisco VIC1457 MLOM (PID HX-MLOM-C25Q-04) andWindows Server Datacenter or Core with Hyper-Vversion 2016 or 2019
CSCvp64140
Release Notes for Cisco HX Data Platform Release 4043
Resolved Caveats in Release 40(1b)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1b)35(2b)Adding Micron 5200 drive to a cluster fails to increasecluster capacity
CSCvo69067
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
ESXi Installation Upgrade Expansion Management
Release Notes for Cisco HX Data Platform Release 4044
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)26(1e)CSCvs28167
Release Notes for Cisco HX Data Platform Release 4045
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
In order to install or complete a node replacement onCisco HyperFlex customers need to download an HXInstaller OVA (Open Virtual Appliance) file in order todeploy a stretched cluster customers additionally needto download a Witness OVA All of the code posted onCCO prior to the posting of release HX 35(2g) wasdiscovered to have expired certificates as of 112619Cisco has re-signed and re-posted OVA files associatedwith HX releases 35(2e) 352(f) 352(g) 40(1a) and40(1b) with updated certificates For other releasesattempts to deploy an OVF template with an expiredOVA will fail with the following error message ldquoTheOVF package is signed with an invalid certificaterdquo
Conditions
If customers are deploying HX 35(2e) 352(f) 352(g)40(1a) or 40(1b) Cisco has re-signed and re-postedOVA files and customers will not experience the problemif they use the patched OVA files Look for a ldquop1rdquo suffixin the OVA filenames which indicates that OVA filehas been fixed
File Name Examples
HX 40(1a) patched OVA file for Cisco HyperFlex DataPlatform Installer for VMware ESXi
Cisco-HX-Data-Platform-Installer-v401a-33028p1-esxova
Cisco HyperFlex Data Platform Stretched ClusterWitness
HyperFlex-Witness-104p1ova
Customers using the OVA files for other HX releasesrefer to the following workaround
Workaround
There are two options to move forward after failing todeploy with an OVA file that is affected (applies to theinstaller and witness OVA files)
Option A - Remove the local manifest file
The manifest file can be deleted so vCenter does notcheck the validity of the certificate
1 Download and extract the OVA file to a localdirectory
2 Remove the mf file
3 Add the remaining files to a new archive and changethe file extension from tar to ova
Release Notes for Cisco HX Data Platform Release 4046
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
Proceed to deploy that newly created OVA file usingldquoDeploy by OVF Templaterdquo in vCenter vCenter will
4
show the file as not having a certificate This isexpected and the deployment should continuewithoutissue
Option B - Remove the local manifest file
Manually deploy with ovftool ndash Use VMwares ovftoolto deploy the OVAwhile bypassing the certificate checkThe ovftool can be downloaded and run on customerscomputer The ovftool also comes pre-installed on HXController VMs This is helpful for node replacementsand cluster expansions
Release Notes for Cisco HX Data Platform Release 4047
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
1 Use ovftool to deploy the OVA file to a datastorewhile raising the --skipManifestcheck switch ForexamplerootSpringpathControllerABCDEFGH~ ovftool--skipManifestCheck -ds=datastorehttpltpath to
ovagtCisco-HX-Data-Platform-Installer-v352c-31725-esxovavirootltIP of management ESX
hostgt
2 TheOVA should be deployed and present in vCenteron the ESXi host previously specified
3 Power on the VM and console into it
4 Login to the VMwith the default usernamepasswordcombination of root Cisco123
5 Set the IP of the VM statically by issuing vietcnetworketh0interface
6 Change iface eth0 inet dhcp to iface eth0 inet staticEach of the following needs to be on their own lineand tab indentedaddress ltdesired ip address of installergt
netmask XXXX
gateway XXXX
ltescgt wq
7 After the file is reviewed and saved restart the VMThe VM should now boot with the desired IP address
8 The first login via the WebGUI (still using defaultusernamepassword combination) will have the userchange the password
9 After the password change the user can begin thedesired installexpandnode replacement activity
40(1a)30(1d)Cluster instability when disks of different sector sizeplaced in HX node
CSCvk17250
40(1a)
35(2c)
35(1a)When logged in using a local HX user account insteadof a Virtual Center account an error message appearsintermittently indicating Virtual Centerunreachable or Resource informationcannot be updated when VC is reachable
CSCvo36198
40(1a)
35(2b)
30(1d)HXDP does not work with EMC RecoverPoint - needsto support VMware API (FSS-Readdir)
CSCvk38003
Release Notes for Cisco HX Data Platform Release 4048
Resolved Caveats in Release 40(1a)
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Resolved inRelease
First ReleaseAffected
SymptomDefect ID
40(1a)
35(2a)
30(1i)
25(1c)
25(1d)
Zookeeper (Exhibitor) process on HyperFlex StorageController VMs may fail to respond to requests whenvarzookeeper has low or no free space
CSCvm90352
40(1a)26(1c)Use Asynchronous consolidation for HX SnapshotsCSCvn02151
40(1a)35(2b)HX Quiesced Snapshot for Backup VendorsCSCvo90713
40(1a)35(2a)The cluster creationcluster expansion workflow stopswith the following error message at the validation step
FIRMWARE-Check UCSC-SAS-M5HD
FIRMWARE-Check UCSC-SAS-M5HD Required00000029000000320000003500000050 Found00000058
Action Needed Update the Controller Firmware toRequired Version
CSCvn17787
35(2b)30(1d)Kernel migration fails when a local datastore is searchedfor in the ESXi host
CSCvn73127
35(2a)26(1b)A node shuts down when two disks are replaced (acaching disk and another capacity disk) where thecapacity disk is inserted first and then the caching diskis inserted
CSCvk46364
Hyper-V
40(1a)35(2a)Cluster expansion may fail with an error code 500 -operation timed out
CSCvn28721
35(2b)35(2a)During upgrade remove the VLAN on team created foruser vSwitch During fresh installation only one VLANtag is set to the vSwitch and team although multipleVLANs were entered
CSCvn54300
35(2b)35(2a)While upgrading a Hyper-V cluster on account of a rarerace condition between the stUpgradeService andZookeeper servers the upgrade orchestration throws anupgrade validation error and the upgrade process isaborted
CSCvn60486
Release Notes for Cisco HX Data Platform Release 4049
Resolved Caveats in Release 40(1a)
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Startrestart EAM service invCenter
When EAM service is not runningvCenter registration is failing
CSCvt55712
35(2a)Contact Cisco TAC if you believeyou are seeing similar conditions
HyperFlex Stretch Cluster saw poorapplication performance due to writelatency cluster remained unhealthy andrebalance was stuck
This defect is currently being used forinvestigative purposes
CSCvt45344
35(2g)Remove log files fromvarlogspringpath folder when thelog files are not getting rotatedStop the relevant service removelarge log files fromvarlogspringpath folder andrestart the service
varstv folder in Controller VM maybecome full
CSCvt36374
40(2b)Restarting the stMgr will initializeit again
In a Cluster configured with DR networkStMgr may not get initialized and stuckin a deadlock while enabling the IPTablerules (can be verified from the stMgrlog)
CSCvt89709
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Click recover for the selectedVM again under the standalonemode This will move the VM torecovered state
When the user recovers a VM protectedin a group it along with other VMs in thegroup move to recovering state instandalonemode The selectedVM shouldmove to recovered state
CSCvs75553
35(2e)Clean up the fail over cluster andthen recreate the fail over clusterNo need to touch the HX storagecluster
Hyper-V When replicated DC was usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2d)40(1a)
Keep cluster utilization as low aspossible To avoid read-only statekeep below 92
HyperFlex clusters storage utilization isabove 92 usable capacity
CSCvr20922
Release Notes for Cisco HX Data Platform Release 4050
Open Caveats in Release 40(2b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1b)NAAll the disks in the node got locked afterrebooting the node
We successfully unlocked usingsed-clientsh -U command but wantedto test with another reboot and driveslocked again
CSCvs74286
35(2g)To recover the VM copy over thedata disks and attach them to thenew VM
After enable encryption HX connectencryption status showing Caution butall of the disks enable encryptionsuccessful
-Self Encrypting Drives Service isrunning on all of the nodes
-There is no error message of encryption
- USB0 interface is up
-All SED disks showing supported 1enabled 1 locked 0
CSCvs93245
40(1b)Enable kernelpanic_on_oops as apersistent configuration This willcause the node to panic and rebootimmediately
A cluster node running HX release40(1b) may hang in the Linux kernelThis is classified as an oops and adeviation from the expected behavior
CSCvs54285
Release Notes for Cisco HX Data Platform Release 4051
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DefectFound inRelease
WorkaroundSymptomDefect ID
40(2a)Enabling K8iscsi stack in Hx clusterCSCvs41324
Release Notes for Cisco HX Data Platform Release 4052
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DefectFound inRelease
WorkaroundSymptomDefect ID
Steps to enable K8 on HX cluster
Perform following steps on ALLthe controller VMs
1 Updateetcinitscvmclientconf toenable tune iscsiEnable totrue
sed -iesiscsiEnable=falseiscsiEnable=trueetcinitscvmclientconf
2 Run following initctlcommand to reloadconfiguration
initctl reload-configuration
3 Restart scvmclient process
stop scvmclient startscvmclient
4 Run following initctlcommand to mount SYSTEMDS
initctl emit --no-waitsystem-datastore-created
5 Verify that iscsiEnable tuneis set to true using thefollowing command
ps -eaf | grep scvmclient |grep -v grep
root 6241 1 1 Dec06 014007optspringpathstorfs-corescvmclient-T logEnabled=false -TlogSyslogEnabled=true -TlogEchoToScreen=false -TstatLoggingToFile=false -TstatLoggingToSyslog=true-TlogDir=varlogspringpath-TnfsBackendServerList=1010748100-T iscsiEnable=true -TiscsiUseAsync=true -T
Release Notes for Cisco HX Data Platform Release 4053
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DefectFound inRelease
WorkaroundSymptomDefect ID
iscsiConfigFilePath=nfsSYSTEMistgtconfps -eaf | grep scvmclient |grep -v grep
35(2e)Following procedure can be usedto check NFS Queue Depth andincrease if needed
rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth256 lt- Low value[rootHXESXI1] vsish -e setvmkModulesnfsclientmnt[DATASTORE]propertiesmaxQDepth 1024[rootHXESXI1] vsish -e getvmkModulesnfsclientmnt[DATASTORE]properties| grep -i maxqdepthmaxQDepth1024 lt- Optimalvalue
This requires an ESXi host rebootto take effect
Please place the node in HyperflexMaintenanceMode and gracefullyreboot the node for the changes tobe applied
HyperFlex Datastore NFS Queue Depthshows as 256 which can lead toperformance (including latency) issues
CSCvr83056
40(1b)This is cosmetic The error will goaway once the cluster returns tohealthy status
When a single node is offline in clusterstcli cluster storage-summary shows twonodes unavailable rootSCVM~ stclicluster storage-summary
CSCvs47735
35(2g)Modify both the root and adminpassword for the SCVMs to be thesame
stcli security password set -uadmin
Cluster expansion fails at Config Installerstage when the root and admin passwordfor the storage controller (SCVM) aredifferent
CSCvs69317
35(2d)NAAfter successfully changing the DNSserver on the HX controller we still cansee the original DNS entry that was addedduring the deployment
CSCvs69154
Release Notes for Cisco HX Data Platform Release 4054
Open Caveats in Release 40(2a)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Customers are strongly encouragedto work with TAC in order toidentify and remediate this issue
1 Change getcluster from 2m to8m inoptspringpathstorfs-mgmtstMgr-10confapplicationconfon all nodes
2 Restart stMgr on all nodes
You may see this error message after afailed upgrade or other task such asattempting to enter a node into HXmaintenance mode
CSCvs53555
40(2a)Run the following command oneach storage controller VM as root
grep -i monitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp sed -iesmonitor_dns_serverstruemonitor_dns_serversfalseoptspringpathhx-diag-toolswatchdog_configjsonampamp grep -imonitor_dns_serversoptspringpathhx-diag-toolswatchdog_configjsonampamp restart watchdog
Receive the following alertevent in HXConnect HX Controller VMHOSTNAME one or more configuredDNS servers not responding
CSCvt13947
35(2b)Delete any empty (Size 0) log filesunder varlogzookeeperversion-2ORvarlogzookeeperstandaloneversion-2
Zookeeper fails to start while Exhibitor isrunning however echo srvr returnsnothing
CSCvs21562
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
40(4e)1 De-commission the server
2 Power drain the server -REMOVE BOTH POWERCORDS ON THE BACK OFTHE SERVER FOR 10SECONDS THENREINSERTPOWERCORDS
3 Re-commission the server
M2 boot disk is missing from serverinventory after upgrade to server firmware40(4e) As a result server fails to boot toOS installed in the M2 disk The issuepersists after re-acknowledgement as wellas de-commission andre-acknowledgement of the server
CSCvs02466
Release Notes for Cisco HX Data Platform Release 4055
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DefectFound inRelease
WorkaroundSymptomDefect ID
35(2e)Re-start the flusher manuallyMany disks taken offline by a fault ormanually resulting in ENOSPC oncluster
CSCvq32721
35(2e)Clean up fail over cluster and thenrecreate the fail over cluster Thereis no need to touch the HX storagecluster
[Hyper-V]When replicated DCwas usedthen during install timeWindows failovercluster was not created successfully
CSCvq38279
35(2b)Manually install upgradescvmclient VIB andmake sure thatit matches with HXDP version
Upgrade from release HX 35(2a) to35(2x) 4X does not upgrade scvmclientRecovery point may not work
CSCvq54992
Management
30(1b)To recover the VM copy over thedata disks and attach them to thenew VM
[HyperV]VMshows up onmultiple nodesCSCvj22992
35(1a)1 Do not change MTU at ESXlevel on running cluster
2 Reset back to the original value
ChangingMTU from 9000 to 1500 on oneESX node causes storfs process to restarton all nodes
CSCvm99150
Release Notes for Cisco HX Data Platform Release 4056
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)After the installation (for newdeployments) or after an upgradeto 401a (for existingdeployments) perform thefollowing steps on all ControllerVMs
1 Edit the following tune files (onall Controller VMs)
optspringpathconfiglfftunes
optspringpathconfigvsi_16tbtunes
2 SetcleanerEnableSegSummaryCleaning
to false
3 After editing the above tunefiles
a SSH in to all the ControllerVMs
b Run storfstool -- -Z
c Run the below command andcheck the values Tune valueshould be true
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=true
d Type below commands andapplied the tunes changesdynamically
echo false gttmpstprocfssystemtunecleanerEnableSegSummaryCleaning
e Verify that tune values arechanged Run below commandValue should be false
cattmpstprocfssystemtunecleanerEnableSegSummaryCleaning
cleanerEnableSegSummaryCleaning=false
f umount tmpstprocfs
A cluster with 8TB or 12TB disk drivesmay experience IO stalls for severalminutes after another node fails in thecluster
CSCvp23718
Release Notes for Cisco HX Data Platform Release 4057
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)1 Update the conf file manuallyto change to the link local addressto
2 Restart scvmclient on thecontroller and ESX
HX 35(2b) FlexVol to HX 40 CSIupgrade does not work FlexVol continuesto work
CSCvp26319
35(2a)Disable rebalance until node healsthen reenable rebalance
7-node cluster healing very slowlybecause rebalance change AMS reqs areblocking healing process
CSCvp62512
35(2a)NAWith moderate cluster storage capacityusage under site failover cluster clustermay experience an all paths (APD) downcondition
CSCvq49412
40(1a)NAThis product includes Third-partySoftware that is affected by thevulnerabilities identified by the followingCommon Vulnerability and Exposures(CVE) IDs
CVE-2016-0762 CVE-2016-6797CVE-2016-6816 CVE-2016-8735CVE-2017-5647 CVE-2017-12615CVE-2017-12616 CVE-2017-12617CVE-2017-7674 CVE-2018-1304CVE-2018-8014 CVE-2018-1336CVE-2018-8034 CVE-2018-11784CVE-2019-0232
CSCvp36364
35(2d)NAstcli cluster info command needs toprovide UCSM VIP address Currently itshows ucsm-hostcom
CSCvq11456
35(2a)No workarounds currently Thestale entries do not affect the dayto day operations of HyperFlex
HyperFlex upgrade validation failedbecause cluster has extraneous stNodes ininternal database
CSCvq32530
35(2a)NAWhen not in maintenancemode all drivesappear in HX Connect and get updated asexpected
When in maintenance mode extra drivesare shown in HX Connect with no slotnumbers
CSCvq42378
35(2b)Contact TAC to cleanup duplicatenode
Nodes duplicated in HX Connect aftercluster recreate
CSCvq61825
Release Notes for Cisco HX Data Platform Release 4058
Open Caveats in Release 40(1b)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)NACurrently HyperFlex installs on Hyper-Vdo not contain the stcli security whitelistcommandset
CSCvq66245
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Install Upgrade Expand
Release Notes for Cisco HX Data Platform Release 4059
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)[Hyper-V] Converged node expansionfails for incoming node with VIC 14xx
CSCvq39523
Release Notes for Cisco HX Data Platform Release 4060
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DefectFound inRelease
WorkaroundSymptomDefect ID
1 Set Jumbo frame inWindowsHypervisor for incomingnode
Get-NetAdapter storage-data-a|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
Get-NetAdapter storage-data-b|Get-NetAdapterAdvancedProperty-RegistryKeywordJumboPacket |Set-NetAdapterAdvancedProperty-RegistryValue 9014
2 Verify all interfaces are setcorrectly to support Jumbo frames
Get-NetAdapter storage |Get-NetAdapterAdvancedProperty| RegistryKeyword -matchjumbo | ft -auto
You should receive the followingmessage
Name DisplayName DisplayValue
RegistryKeyword RegistryValue
---- ----------- ------------
--------------- -------------
vswitch-hx-storage-data Jumbo
Packet 9014 Bytes
JumboPacket 9014
storage-data-b Jumbo Packet
Bytes 9014 JumboPacket
9014
storage-data-a Jumbo Packet
Bytes 9014 JumboPacket
9014
3 Reboot only the incoming nodeor the node being expanded
4 Retry Cluster Expansion in
Release Notes for Cisco HX Data Platform Release 4061
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DefectFound inRelease
WorkaroundSymptomDefect ID
installer (from the point where iterrored out previously)
40(1a)Confirm that both nodes are up andrunning and you have given itsome time to heal (a few hours)before attempting the workaroundIf not healed and failed-back runthe following command on bothcontroller VMs (preferablysimultaneously on both nodes) torestart restarthxRoboController
Two node HyperFlex Edge cluster maynot failback successfully and return tohealthy This can occur if connection toIntersight is highly impaired (egtransaction latencies exceeding 100ms)
CSCvp12241
40(1a)Ensure VC is available beforeperforming DS operations
Datastore createdelete fails when VC isnot available for ESXi HX cluster (regularor stretch or 2N robo)
CSCvp20102
30(1b)
35(1a)
Avoid removal and re-insertion ofdrives that are regarded as goodIf you have performed testing thathas removed and reinserted gooddrives contact Cisco TAC forfurther instructions
Emergency cluster shuts down in HX30(1b) after exiting host from HXMaintenance Mode and storfs panic onother node
CSCvk23212
35(1a)Wait at least 30 minutes for theAD policy to take effect If theissue is not resolved reboot 1 hostat a time usingmaintenancemode
During Hyper-V installation if youchoose to perform constrained delegationlater sometimes it takes an excessiveamount of time to reflect on HX ConnectUI
CSCvm55176
35(2a)Add the ISCSI interface manuallyafter install
Installer process fails when user selectsoption to setup ISCSI interface
CSCvo88857
35(2b)
35(1a)
1 Run the following commandstcli cluster upgrade--components hxdp --clean
2 Re-run the HX upgrade
3 Repeat steps 1 and 2 asnecessary
Stretch cluster upgrade fails randomly dueto Cluster invalid state
CSCvp20230
35(1a)
30(1e)
This error indicates that Windowsfailed to find a domain controllerPlease add a specific IP of DomainController in the Advanced inputof HXInstaller
HX Hyper-V installation fails andHXBootstraplog contains thefollowing message
Unable to find a defaultserver with Active DirectoryWeb Services running
CSCvm53679
Release Notes for Cisco HX Data Platform Release 4062
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DefectFound inRelease
WorkaroundSymptomDefect ID
Management
40(1a)There are two options
A) Wait the 15-20 minutes for theARP entry to time out then thedatastore will become mountedagain
B) Manually reset the ARP entryby following these steps
1 Check the current eth10 macaddress from both HyperFlexController VMs and node theIPMac
Run ifconfig eth10 from theshell of the controller VM
2 On the node where the datastoreis unavailable check the macaddress entries for the above IPaddresses in the ESXi ARP cacheusing the following command
esxcli network ip neighbor list
3 If the IP address is assigned tothe incorrect mac address purgethe entry from the ARP table asshown below
esxcli network ip neighbor
remove -a ltIP_addressgt -v 4
After 2 node network partition healsdatastore on the isolated node is notavailable for 15 minutes
CSCvp98910
35(2b)If upgrading to 35(2b) or 40I(1a)enabling the RecoverPoint featuremay require you to perform arolling node by node HXMaintenance Mode in the clusterEnsure that the cluster is healthyand can tolerate one node failure(for 3 or 4 node clusters) and 2node failures (for 5 or greater than5 node clusters)
Deploy of EMC RecoverPoint fails witherror Failed findingrepository device in thevRPA view
CSCvp21417
Release Notes for Cisco HX Data Platform Release 4063
Open Caveats in Release 40(1a)
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DefectFound inRelease
WorkaroundSymptomDefect ID
40(1a)
35(2b)
Remove the Micron 5200 drivesfrom the system Upgrade toRelease 35(2b) then followdisk-add expansion workflow
In the event of adding unsupportedMicron5200 drives to an HX cluster and thenupgrading HX to a release that supportsthem the drives could get locked if thecluster has remote security enabled(during certain cases like continuousreboot of controller VMs)
CSCvo89507
35(1a)NAHX Datastore usage is higher than thecombined usage of the VMs on thedatastore
CSCvn76916
35(1a)Retake the snapshotVM powers off during backup VMsnapshot
CSCvo83276
35(1a)
30(1e)
1 Verify if the interface is up andif you can ping the loopbackinterface
ifconfig -aping 127001
2 Bring up the loopback interfaceip link set lo up
3 Check that the service isrunningstatus scvmclientstatusstorfs
4 Start the followingservicesstart scvmclientstartstorfs
HX node keeps crashing after node isrestarted
CSCvn11045
35(2b)Use the command stcli servicessch show instead
Cluster info shows Smart call home isenabled even though it is disabled
CSCvp09978
Related Caveats
Resolved inRelease
Defect Found inRelease
SymptomDefect ID
OpenCisco IMC40(1a)
When attempting to install ESXi 65 or 67 from a CIMCmounted ISO with an embedded kickstart file theinstallationmay fail when reading the embeddedKSCFGfile In the ESXi installer a popup error will state
Could not open file ltpathgtKSCFG
CSCvq41985
Release Notes for Cisco HX Data Platform Release 4064
Related Caveats
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
Related Documentation
DescriptionDocument
Provides an editable file for gathering required configurationinformation prior to starting an installation This checklist must be filledout and returned to a Cisco account team
Preinstallation Checklist
Provides detailed information about Day 0 configuration of HyperFlexSystems and related post cluster configuration tasks It also describeshow to set up multiple HX clusters expand an HX cluster set up amixed HX cluster and attach external storage
Installation Guide for VMwareESXi
Provides installation and configuration procedures for HyperFlexStretched cluster enabling you to deploy an Active-Active disasteravoidance solution for mission critical workloads
Stretched Cluster Guide
Provides installation and configuration procedures on how to installand configure Cisco HyperFlex Systems on Microsoft Hyper-V
Installation Guide on MicrosoftHyper-V
Provides deployment procedures for HyperFlex Edge designed to bringhyperconvergence to remote and branch office (ROBO) and edgeenvironments
Edge Deployment Guide
Provides information about how to manage and monitor the clusterencryption data protection (replication and recovery) ReadyClonesNative snapshots and user management Interfaces includeHXConnectHX Data Platform Plug-in and the stcli commands
Administration Guide
Provides information about how to manage and monitor the Hyper-Vcluster encryption data protection (replication and recovery)ReadyClones Hyper-V Checkpoints and user management Interfacesinclude Cisco HyperFlex Systems and the hxcli commands
Administration Guide for Hyper-V
Provides information about HyperFlex storage integration forKubernetes information on Kubernetes support in HyperFlex Connectand instructions on how to configure CiscoHyperFlex Container StorageInterface (CSI) storage integration for both the Cisco container platformand the RedHat OpenShift container platform
Administration Guide forKubernetes
Provides installation configuration and deployment procedures for aHyperFlex system to connect to CitrixWorkspaces and associated CitrixCloud subscription services such as Citrix Virtual Apps and DesktopsServices The Citrix Ready HCIWorkspace Appliance program enablesa Cisco HyperFlex System deployed on Microsoft Hyper-V to connectto Citrix Cloud
Administration Guide for CitrixWorkspace Appliance
Provides installation configuration and deployment procedures forHyperFlex Intersight designed to deliver secure infrastructuremanagement anywhere from the cloud
HyperFlex Intersight InstallationGuide
Release Notes for Cisco HX Data Platform Release 4065
Related Documentation
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-
DescriptionDocument
Provides information on how to upgrade an existing installation of CiscoHX Data Platform upgrade guidelines and information about variousupgrade tasks
Upgrade Guide
Provides information about HyperFlex Systems specific network andexternal storage management tasks
Network and External StorageManagement Guide
Provides CLI reference information for HX Data Platform stcli
commandsCommand Line Interface (CLI)Guide
Provides information on how to use the Cisco PowerShell CiscoHXPowerCLI cmdlets for Data Protection
Cisco HyperFlex PowerShellCmdlets for Disaster Recovery
Provides information related to REST APIs that enable externalapplications to interface directly with the Cisco HyperFlex managementplane
REST API Getting Started Guide
REST API Reference
Provides troubleshooting for installation configuration to configurationand to configuration In addition this guide provides information aboutunderstanding system events errors Smart Call Home and Ciscosupport
Troubleshooting Guide
Provides independent knowledge base articlesTechNotes
Provides information on recommended FIServer firmwareRelease Notes for UCS ManagerRelease 40
Release Notes for Cisco HX Data Platform Release 4066
Related Documentation
- Release Notes for Cisco HX Data Platform Release 40
- Introduction
-
- Revision History
- New Features
- Supported Versions and System Requirements
-
- HyperFlex Software Versions
- HyperFlex Edge and Firmware Compatibility Matrix for 4x Deployments
- HyperFlex Licensing
- HX Data Platform Software Versions for HyperFlex Witness Node
- Software Requirements for VMware ESXi
- Software Requirements for Microsoft Hyper-V
- Browser Recommendations
-
- Cisco HX Data Platform Storage Cluster Specifications
- Guidelines and Limitations
- Mixed Cluster Expansion Guidelines
- Security Fixes
- Resolved Caveats in Release 40(2b)
- Resolved Caveats in Release 40(2a)
- Resolved Caveats in Release 40(1b)
- Resolved Caveats in Release 40(1a)
- Open Caveats in Release 40(2b)
- Open Caveats in Release 40(2a)
- Open Caveats in Release 40(1b)
- Open Caveats in Release 40(1a)
- Related Caveats
- Related Documentation
-