August 27, 2018 Sam Siewert
CS317File and Database Systems
Lecture 1 - Introduction
Dr. Sam SiewertJohnson Space Center– Shuttle Ascent and Entry GN&C, Mission Control Center
JPL AI Group, CU Space Grant and Ball Aerospace
CU Boulder Adjunct Professor, CTO, Architect, Developer/Engineer in Local Start-ups and Intel Architecture / Labs
U. of Alaska, Assistant Professor, Computer Systems Engineering, Alaska Space Grant, DHS ADAC
Embry Riddle Prescott, Assistant Professor, Computer, Electrical and Software Engineering, ARI Drone Net
Sam Siewert 2
https://en.wikipedia.org/wiki/Intel
Preparation for CS 317 - SE’sCS118 (Python), CS125 (C), CS315 (Data Structures)Goes well with OS (Linux)C++ not required, but OO will be discussed
Sam Siewert 3
File systems - Linux
Physical DBMS - C
Logical DBMS - SQL
Python MySQLConnectors (Apps)
C Connectors
[C++ - optional]
[JavaScript/PHP]
Preparation for CS 317 - CybersecuritySpring CS 317 Section with Cybersecurity Emphasis
CS118 (Python), CS125 (C), CS315 (Data Structures)Goes well with OS (Linux)C++ not required, but OO will be discussed
Database Security Notes
Sam Siewert 4
Fall - Emphasis on Design for Security
Secure Connector Apps (SQL Injection)Secure Design - Stored Programs, Views,DBA Access Control (Grants, Priveleges)
Sam Siewert 5
Related Industry BackgroundGeneral Experience (~25 Years in Embedded and Scalable Systems)– 12 Years NASA JSC, NASA JPL, CU, Ball
Aerospace– 12+ Years Commercial Telecomm,
Storage/Networks, Embedded, Digital Video
Software Engineering– NASA Johnson and JPL (Shuttle Ascent/Entry
Guidance, Deep Space)– Intel, Emulex, Start-ups
CTO– RAID, HPC– iSCSI, Fiber Channel, Infiniband– Large Boxes of Disk Drives - JBODs– Large Boxes of SSDs - JBOF?
Consulting– Graphics, Storage and Networking– Advanced RAID and Erasure Codes
RAID
SAS
File and DB Systems Experience2002-2006 – Emulex (Intel), Chip-down Fiber Channel
2006-2010 – Atrato Inc. (Start-up), Scalable HDD/SSD Hybrid RAID Systems (10GE, 4/8G Fiber Channel)
2010-12 – Intel Corporation – Beyond Software RAID Research
Low-Level! - Block Layer Below Oracle, MS SQL, MySQL
Self-Taught on MySQL (Easy to Learn, Hard to Master)
Sam Siewert 6
Learning ObjectivesIntroduction - Block Storage, Structured and Unstructured Data
– Block Storage Devices [Disk Drives, SSD, Persistent Memory]– Split between Files and Databases [Integration of Both]
DML - Data Manipulation using SQL (ISO Standard)
DDL - Data Definition Language, Database Design with SQL– Logical Design [Schema]– Physical Design [Hosting DB]
Theory of Databases– Relational, – OODBMS (SQL Extensions, C++ Alternatives)– NoSQL
DBA - Database Administration and Security
File and Database Systems– Physical Implementation and Scaling [Block Partition, File], – Indexing [ISAM, B Tree, B+ Tree, R-Tree], – Network Access [Connectors], – Web Front Ends and Connectors (C, Python, C++, JavaScript, PHP)
Sam Siewert 7
Current Research - Data AnalyticsBlockchain for UAS, Machine Learning for UASERAU ICARUS Drone Net [2016 – present]5th Big Data Silicon Valley, Drone Net, NASA UTM
Sam Siewert 8
Machine Vision
Machine Learning
Big Data Analytics
GP-GPUs, FPGA
Google TPU
NAS Storage
Structured & Unstructured Data
Course Goals and OutlineTextbook - Database Systems: A Practical Approach to Design, Implementation, and Management, 6th Edition, Thomas Connolly and Carolyn Begg, ISBN 978-0-13-294326-0 (Amazon, Publisher)
… introduces the theory behind databases in a concise yet comprehensive manner, providing database design methodology that can be used by both technical and non-technical readers. The methodology for relational Database Management Systems is presented in simple, step-by-step instructions in conjunction with a realistic worked example using three explicit phases—conceptual, logical, and physical database design.
http://mercury.pr.erau.edu/~siewerts/cs317/
Current Syllabus
Sam Siewert 9
New Textbook!Reference Only forSecurity Considerations
Primary TextbookProfessional IT Referencefor Oracle Enterprise Systems
Why SQL and MySQLWide Use, Partially Open Source (Alternate – MariaDB)RDBMS Majority Market Share – DB-Engines RankingAlternatives to R-DBMS1. NoSQL - MongoDB, Redis, Cassandra2. Distributed Data - Blockchain (Linux Foundation - Hyperledger)3. OODBMS (C++ Connectors)
Sam Siewert 10
File System and DBMS CodeTake Poll on Who has had CS125 [C programming], CS225 [C++ programming], CS315 [Data structures]
ALL must Learn SQL and Use PRCLab1 Web Interface and Command Line Interface
Assignment #5 Options [Teams]– Dig Deeper into DBA SQL Security [Non C/C++ Programmer
Option]– DBE Design to Deploy a DDL Schema and Populate with DML Test
Data – Build a Front End [C, Python, JavaScript, PHP]– Alternative DB - Blockchain, NoSQL (SE Workstation), Mobile
(SQLite)
ALL must Learn R-DBMS and Logical Design Concepts
Sam Siewert 11
PRClab1 – Linux for SQL and COption #1 – Use PRClab1, prclab1.erau.edu via SSH– Unix Account LDAP - Ernie login– Recommend MobaXterm access– SQL and C Code Development (GCC, Make, etc.) - Linux Help – MySQL Workbench (on STEM Room 110 and Bldg #16 Lab)– http://prclab1.erau.edu/phpmyadmin/ (PHPAdmin login)– IT Support [email protected] – For Help with PRClab1
login, etc.
Option #2 – Use X2Go and SE Workstation– Must Have Windows, Macintosh or Linux Personal PC and install
X2Go– SE Workstation Access Instructions - Ryan Sutton– Maria DB instead of MySQL (possible)– Final alternative project with Maria, NoSQL, or Blockchain
Sam Siewert 12
Sam Siewert 13
Why Work with Linux and Virtual Box…From Mobiles to Super-Computing to Datacenters
From Android Mobiles to GIS and Digital Video Services
Huge Value in Open Source Drivers, Tools, and Applications – Speeds Up Time to MarketOracle Virtual Box – Great Cross OS Test Environment
http://www.top500.org/
Tianhe – 33+ PflopsGoogleEmbedded/Mobile
How We’ll Do It1/3 Knowledge, Concepts, Theory – Lectures/Reading– Lectures related to Connolly-Begg Textbook and Instructor’s
Experience– Logical and Physical DBMS– Review of File systems and Key Differences with DBMS– LAMP – Linux, Apache, MySQL, PHP– Discussions
1/3 Practice – PRClab1 or SE Workstation LAMP– SQL, C code, Python, [JavaScript and PHP if you wish]– Building, Modifying, Querying and Using MySQL (MariaDB)– Logical RDBMS Design, Implementation and Management– Physical RDBMS Data structures (B-tree), Physical Ext-n I-node
structures and debugfs
1/3 Project [DBA, DBE]– Group Project to Build a Significant Database on PRClab– Final Assignment
Sam Siewert 14
Sam Siewert 15
AdministriviaIntroductions– Instructor (Office Hours) – Office
Hours– Students (Introductions) – Please do
Collaborate, but cite well!– Policies
Mercury and CANVAS– CANVAS Assignment Management– Access via ERNIE -
https://ernie.erau.edu– CS 317 Mercury Website
Course Information– Attendance & E-mail list (please sign
up on sheet being passed around)– Mercury Lecture Notes– Will post Assignments on CANVAS
Must have PRClab1 account -prclab1.erau.edu, http://prclab1.erau.edu/phpmyadmin/
PHP WebAccess
SQL CLIAccess
25 August 2014 Sam Siewert
MySQL on LinuxLAMP vs. Connector App
onFile system / Block Storage
Introduction Session(File System vs. DBMS)
Linux OS
SAN, NAS
MySQL
PHP
Apache
Web Client
C, Python App
4-tier PRClab with MyPHP LAMPFirst tier– Your client Web browser
Second tier– Apache Web server
Third tier– PHP interface between Web
server and MySQL
Fourth tier– MySQL / Linux
Sam Siewert 17
ApacheWeb Server
AdminerPHP
MySQL
Storage
3-tier PRClab with Connector AppFirst tier– Your App - GUI, Mobile, CLI
Second tier– Connector tunnel/path
between App and MySQL
Third tier– MySQL / Linux
Sam Siewert 18
PRClab1App
Connector(C, C++, Python)
MySQL
Storage
File System Concepts – Review?Name SpaceMeta-DataFile-DataI-nodesBlocks with Byte Offsets Belong to Name Space Files
Sam Siewert 19
Volume Use by File and Folder
Sam Siewert 20
Make a File Systemmkfs –t ext3 /dev/sdb1Install Name Space and Metadata on Volume
Sam Siewert 21
Use a File SystemBlock Access via dd in Linux – e.g. Clone VolumeName Space Access via Shell or Browser
Sam Siewert 22
Sam Siewert 23
MySQL Command Line and PHPAdminWhy and How is a RDBMS Different?
Login to MySQL via Webhttp://prclab.pr.erau.edu/adminer.php
Sign up for Unix account today!!!ExploreRead Tutorial Docs -http://dev.mysql.com/doc/index-topic.html
Login to MySQL via SSH Shell (MobaXterm) - TA/DBA Account Creation E-mail coming …
In Case You Want More (NOT REQUIRED)Install Oracle Virtual BoxInstall Centos or Ubuntu LTSInstall MySQLInstall Apache and PHPSetup PHPAdmin on localhost
Next Time …DBMS vs. File System (read DBMS Chapter #1) – Come to Class Prepared to Discuss– Why is DBMS Better then File System– Why do we Use File Systems still?– What is NoSQL? (Not in Book, http://en.wikipedia.org/wiki/Nosql)– Structured vs. Unstructured Data? – For Discussion
Assignment #1 Discussion– I will Post Every Other Tuesday, We’ll Discuss, Due Following
Week on Friday– Late Assignments – 10% Penalty for Monday Turn-in, After
Monday, only with Instructor Permission
Sam Siewert 24
Top Related