Post on 25-Dec-2015
www.egi.eu
EGI-Engage
www.egi.eu
Recent Experiences in Operational Security:
Incident prevention and incident handling in the EGI and WLCG infrastructure
Dr Linda Cornwall, STFC.
HEPiX Spring 2015
24th March 2015 Linda Cornwall, STFC 1
www.egi.eu
WLCG and EGI
• The (Worldwide) LHC Computing Grid and The European EGI Infrastructure share a lot of the same resources
• Also share Security teams and activities
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford2
www.egi.eu
Contents
• Incident Prevention • Policy definition• Vulnerability handling• Security monitoring
• Incident handling and incidents from the last year
• Evolving the work
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford3
www.egi.eu
Security Incident Prevention
• Far more work goes into preventing incidents than handling them• Security Policy definition• Software Security, especially Software
Vulnerability handling• Security monitoring - monitoring for known
vulnerabilities and insecure configuration
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford4
www.egi.eu
Security Policy Definition
• Security Policy definition is carried out by the EGI Security Policy Group (SPG)• Defines the behaviour expected from NGIs, Sites,
Users and other participants to maintain a beneficial and effective working environment
• Output is various policy documents• Parties read and sign, so that they know and
understand what they should and should not do• List of policy docs at:
https://wiki.egi.eu/wiki/SPG:Documents
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford5
www.egi.eu
Minimizing vulnerabilities in the infrastructure
• Handling vulnerabilities found/reported• Main activity of the EGI Software Vulnerability
Group
• Assessing software for vulnerabilities• Formally and informally
• Preventing new vulnerabilities being introduced • Developer education, awareness
• Considering new software to be used in the infrastructure
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford6
www.egi.eu
Software Vulnerability Handling
• Approved procedure (Under Revision)https://documents.egi.eu/public/ShowDocument?docid=717
• Anyone may report an issue• By e-mail to report-vulnerability@egi.eu• This may be because they have found it in software• Or it may be that it has been announced
• If it has not been announced, SVG contacts the software provider and the software provider investigates (with SVG member, reporter as appropriate)
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford7
www.egi.eu
Relevance and Risk
• The relevance to EGI is considered, and what affect it could have
• Then it is risk assessed, and put in 1 of 4 categories• ‘Critical’, ‘High’, ‘Moderate’ or ‘Low’
• If it has not been fixed, target date set• ‘Critical’ 3 days, ‘High’ 6 weeks, ‘Moderate’ 4
months, ‘Low’ 1 year
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford8
www.egi.eu
Advisory issued
• An advisory is issued when vulnerability is fixed if EGI SVG IS the main handler of vulnerabilities for this software, or software is in EGI UMD regardless of the risk.• E.g. Grid Middleware, tools developed in EGI
and collaborating projects
• If EGI is NOT the main handler, e.g.linux advisory only issued if ‘High’ or ‘Critical’
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford9
www.egi.eu
SVG message – if you find a vulnerability
If it is NOT public knowledge •DO NOT
• Discuss on a mailing list – especially one with an open subscription policy or which is archived publically
• Post information on a web page• Publicise in any way without agreement of SVG
•DO report to SVG via
report-vulnerability@egi.eu
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford10
www.egi.eu
High and Critical Vulnerabilities monitored
• Sites are monitored for ‘High’ and ‘Critical vulnerabilities.
• EGI CSIRT chases sites which are exposing ‘Critical’ vulnerabilities
• Sites may get suspended if they expose critical vulnerabilities and don’t respond
• Respond if asked to by IRTF/CSIRT
• For ‘High’ risk, up to the local NGIs.
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford11
www.egi.eu
Vulnerabilities reported during last year
• 42 new entries in vulnerability tracker (RT)• 12 concerned Grid Middleware - 2 critical
(1related to heartbleed, 1 related to perfsonar/cacti) 4 high
• 16 Linux – 3 critical (heartbleed, Shellshock, Kernel) 5 high
• 4 Cloud enabling – 3 high• 6 VO software – 3 high • Others include 1 high
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford12
www.egi.eu
Changing types of Vulnerabilities
• Until about 1 year ago most vulnerabilities concerned Grid Middleware
• Now more concerning VO specific software• Including Data Protection issues • VOs take it into their head to ‘monitor’ activities
in a way that is traceable back to user
• Cloud specific software• Less knowledge about this
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford13
www.egi.eu
Incident Handling
• Approved Incident handling procedurehttps://documents.egi.eu/public/ShowDocument?docid=710
• Incidents are handed by the Incident Response Task Force.
• Fortunately there are not many • Incident prevention is quite successful
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford14
www.egi.eu
Incidents during last year (8)
• Primecoin mining (Policy violation)• Open Hostkey leaking private information• User cert mis-use• Fed Cloud incident
• Due to bad endorsed VM• UI compromised (4 user IDs compromized)• Shellshock related compromises to Perfsonar nodes
(multiple sites)• Compromise due to port left open• DDoS to some EGI services
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford15
www.egi.eu
Evolving the Security Work
• Evolving the security work is necessary due to e.g.• The EGI federated Cloud
• Changing responsibility model• Changing technology
• Long Tail of Science• Different trust model
Have some H2020 funding for EGI engage to carry out this evolution
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford16
www.egi.eu
Policy documents under revision
• Getting rid of ‘Grid’• Policies apply to all technology and services
• Acceptable use policy• External draft – request for feedback and
comments
https://wiki.egi.eu/wiki/SPG:Drafts:Acceptable_Use_Policy_March_2015
• Security Policy for the endorsement and operation of Virtual Machine images• Especially for Fed Cloud experience
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford17
www.egi.eu
New policy documents
• Data Protection Policy• Formerly only had “Grid Policy on the handling
of User Level Job accounting data• Finding Data protection policy needed as User
level data is being monitored and exposed inappropriately.
• Long Tail of Science Policy• Related to allowing access other than by large
VOs, IGTF certificates• User sub-proxy.
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford18
www.egi.eu
Vulnerability handling evolution
• Now more software is coming into use where SVG members have no knowledge• New members of SVG who know about cloud
software, especially tools written within the community
• ‘Expert’ contact for all software Cloud enabling software deployed in the Fed Cloud
• VO software – assume VO security contact is responsible and know who to contact
• No more than 2 steps to the right person.
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford19
www.egi.eu
Software security checking
• For some community cloud enabling software have a detailed ‘Technology provider’ questionnaire
• For other software propose something simpler:-- • License details• How long will it be under security support?• How are security problems reported? • Are security problems announced? • Check compliance with Data Protection policy• Some other simple technical checks – e.g. is user input
is validated, bad constructs – not obviously bad
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford20
www.egi.eu
Incident response evolution
• Changing responsibility model in the cloud will mean changes to incident response.
• A lot of work is going on including traceability – See Ian Collier’s talk
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford21
www.egi.eu
Questions??
24th March 2015Linda Cornwall, STFC.
HEPiX Spring conference, Oxford22
www.egi.eu24th March 2015
Linda Cornwall, STFC. HEPiX Spring conference, Oxford
23