IT Data Visualization - Sumit 2008
-
Upload
raffael-marty -
Category
Technology
-
view
697 -
download
1
Transcript of IT Data Visualization - Sumit 2008
![Page 1: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/1.jpg)
IT Data Visualization
Raffael Marty GCIA CISSPChief Security Strategist Splunkgt
SUMIT Michigan - October lsquo08
bull Chief Security Strategist Splunkgt
bull Looked at logsIT data for over 10 years
- IBM Research
- Conference boards committees
bull Presenting around the world on SecViz
bull Passion for Visualization
- httpsecvizorg
- httpafterglowsourceforgenet
Raffael Marty
Applied Security VisualizationPaperback 552 pages
Publisher Addison Wesley (August 2008)ISBN 0321510100
Agendabull IT Data Visualization
- Security Visualization Dichotomy
- Research Dichotomy
bull IT Data Management
- A shifted crime landscape
bull Perimeter Threat
bull Insider Threat
bull Security Visualization Community
3
Visualization is a more effective way of IT data management and
analysis
Visualization Questionsbull Who analyzes logs
bull Who uses visualization for log analysis
bull Who has used DAVIX
bull Have you heard of SecVizorg
bull What tools are you using for log analysis
4
IT Data Visualization
Applied Security Visualization Chapter 3
What is Visualization
6
A picture is worth a thousand log records
Generate a picture from IT data
Inspire
Pose a New Question
Explore and Discover
Support Decisions
Communicate Information
Increase Efficiency
Answer a Question
Information Visualization Process
7
Capture Process Visualize
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 2: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/2.jpg)
bull Chief Security Strategist Splunkgt
bull Looked at logsIT data for over 10 years
- IBM Research
- Conference boards committees
bull Presenting around the world on SecViz
bull Passion for Visualization
- httpsecvizorg
- httpafterglowsourceforgenet
Raffael Marty
Applied Security VisualizationPaperback 552 pages
Publisher Addison Wesley (August 2008)ISBN 0321510100
Agendabull IT Data Visualization
- Security Visualization Dichotomy
- Research Dichotomy
bull IT Data Management
- A shifted crime landscape
bull Perimeter Threat
bull Insider Threat
bull Security Visualization Community
3
Visualization is a more effective way of IT data management and
analysis
Visualization Questionsbull Who analyzes logs
bull Who uses visualization for log analysis
bull Who has used DAVIX
bull Have you heard of SecVizorg
bull What tools are you using for log analysis
4
IT Data Visualization
Applied Security Visualization Chapter 3
What is Visualization
6
A picture is worth a thousand log records
Generate a picture from IT data
Inspire
Pose a New Question
Explore and Discover
Support Decisions
Communicate Information
Increase Efficiency
Answer a Question
Information Visualization Process
7
Capture Process Visualize
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 3: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/3.jpg)
Agendabull IT Data Visualization
- Security Visualization Dichotomy
- Research Dichotomy
bull IT Data Management
- A shifted crime landscape
bull Perimeter Threat
bull Insider Threat
bull Security Visualization Community
3
Visualization is a more effective way of IT data management and
analysis
Visualization Questionsbull Who analyzes logs
bull Who uses visualization for log analysis
bull Who has used DAVIX
bull Have you heard of SecVizorg
bull What tools are you using for log analysis
4
IT Data Visualization
Applied Security Visualization Chapter 3
What is Visualization
6
A picture is worth a thousand log records
Generate a picture from IT data
Inspire
Pose a New Question
Explore and Discover
Support Decisions
Communicate Information
Increase Efficiency
Answer a Question
Information Visualization Process
7
Capture Process Visualize
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 4: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/4.jpg)
Visualization Questionsbull Who analyzes logs
bull Who uses visualization for log analysis
bull Who has used DAVIX
bull Have you heard of SecVizorg
bull What tools are you using for log analysis
4
IT Data Visualization
Applied Security Visualization Chapter 3
What is Visualization
6
A picture is worth a thousand log records
Generate a picture from IT data
Inspire
Pose a New Question
Explore and Discover
Support Decisions
Communicate Information
Increase Efficiency
Answer a Question
Information Visualization Process
7
Capture Process Visualize
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 5: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/5.jpg)
IT Data Visualization
Applied Security Visualization Chapter 3
What is Visualization
6
A picture is worth a thousand log records
Generate a picture from IT data
Inspire
Pose a New Question
Explore and Discover
Support Decisions
Communicate Information
Increase Efficiency
Answer a Question
Information Visualization Process
7
Capture Process Visualize
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 6: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/6.jpg)
What is Visualization
6
A picture is worth a thousand log records
Generate a picture from IT data
Inspire
Pose a New Question
Explore and Discover
Support Decisions
Communicate Information
Increase Efficiency
Answer a Question
Information Visualization Process
7
Capture Process Visualize
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 7: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/7.jpg)
Information Visualization Process
7
Capture Process Visualize
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 8: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/8.jpg)
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 9: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/9.jpg)
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 10: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/10.jpg)
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 11: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/11.jpg)
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 12: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/12.jpg)
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 13: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/13.jpg)
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 14: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/14.jpg)
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 15: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/15.jpg)
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 16: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/16.jpg)
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 17: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/17.jpg)
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 18: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/18.jpg)
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 19: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/19.jpg)
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 20: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/20.jpg)
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 21: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/21.jpg)
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 22: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/22.jpg)
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 23: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/23.jpg)
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 24: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/24.jpg)
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 25: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/25.jpg)
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 26: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/26.jpg)
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 27: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/27.jpg)
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 28: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/28.jpg)
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 29: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/29.jpg)
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 30: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/30.jpg)
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 31: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/31.jpg)
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 32: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/32.jpg)
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 33: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/33.jpg)
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 34: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/34.jpg)
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
![Page 35: IT Data Visualization - Sumit 2008](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4fad0bb61eb714c8b45ac/html5/thumbnails/35.jpg)
Thank You
raffy splunk com