Summit 16: Carrier Grade Testing Integration
-
Upload
opnfv -
Category
Technology
-
view
249 -
download
0
Transcript of Summit 16: Carrier Grade Testing Integration
![Page 1: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/1.jpg)
June 20–23, 2016 | Berlin, Germany
![Page 2: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/2.jpg)
Carrier Grade Testing Integration
J.Lausuch, Ericsson
M.Richomme ,Orange
![Page 3: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/3.jpg)
Carrier grade?
![Page 4: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/4.jpg)
Carrier grade?
![Page 5: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/5.jpg)
Carrier grade?
Not a single definition
The power of the IT cloud meeting Telco constraintsOr
How to 99.999% with 99.999% subsystems
![Page 6: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/6.jpg)
¿Carrier grade?
A question
of
confidence ?
Robust
Manageable
Scalable
Agile
Legalconstraints
Secure Efficient
Monitoring/SupervisionTroubleshootingBackup/restoreLegacy support
HAGeographic redundancyDowntime / SLA
Resources optimizationIntelligent placement
VNFs easy integrationMulti Mano stacksSoft/hard update
For our customersFor the system
Critical real-time applications QoS control
Legal InterceptionData retentionEmergency services
![Page 7: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/7.jpg)
Carrier grade?
![Page 8: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/8.jpg)
Our first answers
![Page 9: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/9.jpg)
OPNFV Testing community
Sanity testing
Performance testing
Maintainability testing
Reliability testing
System testing
Black box testing
Regression testingIntegration testing
Security testing
Acceptance testing
Unit testing
Load testing
Compatibility testing Recovery testing
Usability testing
Comparison testing
Stress testingEnd-to-end testing
Scalability testing
Compliance testing
Efficiency testing
Functional testing
![Page 10: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/10.jpg)
OPNFV Testing community
Functional testing
Performance testing
Feature testing
![Page 11: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/11.jpg)
OPNFV Testing community
![Page 12: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/12.jpg)
From Telco issue to OPNFV projects
Fault management
Resource management
Smooth Upgrade
Multisite
HA
Doctor
Promise
Escalator
Multisite
HA
![Page 13: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/13.jpg)
Focus on feature projects
Open vSwitch
SFC
BGPVPN
Copper…
![Page 14: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/14.jpg)
Focus on feature projects
Problem detected
OPNFV Feature Project created
Dev in Upstream project 1
Dev in Upstream project N
Feature integrated in
OPNFV scenario(s)
Feature tested in CI
Feature integrated in
OPNFV releaseInternal devFeature tests dev
![Page 15: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/15.jpg)
Focus on CI
automation, tooling and software infrastructure support for OPNFV projects
Why?
What for?
What? software development infrastructure
repeatability and stability control
![Page 16: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/16.jpg)
Focus on Infrastructure
Define the min. HW requirements
to deploy and test OPNFV.
Available to the community for
development and automation.
Used to validate OPNFV releases.
![Page 17: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/17.jpg)
The challenges
![Page 18: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/18.jpg)
Beyond CI, repeatability
Automating tests is not enough
Understanding behavior changes is mandatory to
Save time (not run systematically all the tests)
Adapt the test strategy according to the environment
![Page 19: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/19.jpg)
More features, more scenarios….
Problems:More Features to be tested…
More Scenarios to be tested…
Days are limited to 24h…
Hardware resources for tests may increase but not exponentially
Solutions:Reduce the number of tests
Select/promote Telco grade scenarios
![Page 20: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/20.jpg)
Telco testing means:
repeat the same test again and again.
stress the system over days/weeks.
KPI compliance.
Reliability
Stability
Performant
![Page 21: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/21.jpg)
Here comes MANO…
NFVI and VIM Telco grade foundations are still questionable…still lots to do
MANO game is very open (descriptors, pragmatic versus formal approach,..)
Lots of Open Source MANO initiatives• More tests, more combinations, …
• Time needed before possible momentum
![Page 22: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/22.jpg)
What about O&M?
• O&M tests not addressed yet in OPNFV
• Several possible options (nagios, centreon, zabix, skydive,…)
• Shall we test the integration of such tools as part of OPNFV
![Page 23: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/23.jpg)
Conclusions
![Page 24: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/24.jpg)
Thank you
![Page 25: Summit 16: Carrier Grade Testing Integration](https://reader031.fdocuments.us/reader031/viewer/2022030305/587097cb1a28ab412b8b6d19/html5/thumbnails/25.jpg)
Credits
• Earthquake by Irfan Syler, FreeImages.com Content License
• FIREMEN AT WORK II by Ossian Engmark, FreeImages.com Content License