ITEC400 Week Three

ITEC400 Week Three

ITEC400 Week ThreeRobert D’Andrea

AgendaAdministrative Issues Link of the Week Review Week Two Information This Week’s Expected Outcomes Next Lab Assignment Break-Out Problems Upcoming Deadlines Lab Assistance, Questions, and Answers

Administrative issuesCall and/or email me if you don’t understand an assignment, procedure, or just need a pointer (614.519.5853).

Link of the weekAPA Writing Style

APA Style writing guidelines will be utilized for all lab assignment reports .

APA Style Sixth Editionhttp://www.apastyle.org/learn/APA Style Writing Workshophttp://www.franklin.edu/student-services/student-learning-center/academic-support/workshops.html

Grading will focus mainly on capitalization, punctuation, grammar, and citation.

Link of the weekLinux Forums Web Site

This web site allows individuals to post questions about Fedora Linux and Redhat Linux. The people that maintain this site are knowledgeable users of these operating systems. If your experiencing problems with commands or just want to learn more about these systems, you can find this type of information at this site.

http://www.linuxforums.org/forum/redhat-fedora-linux-help/73994-system-commands-not-working-unless-sbin.html

Course expected outcome Week 3

Upon successful completion of this module, the student will be able to:

Authors of UNIXCreate scripts using shell/Perl variables and

program control flow. Authors of AWKUse redirection and pipes to combine scripts

and executables. Use man page system and find script tools.

Discuss Perl Language

UNIX Operating SystemAuthors of UNIXDennis RitchieComputer ScientistDennis Ritchie was an American computer scientist. He

created the C programming language and, with long-time colleague Ken Thompson, the Unix operating system.

Born: September 9, 1941, Bronxville, NYDied: October 12, 2011, Berkeley Heights, NJEducation: Harvard UniversityFields: Computer ScienceBooks: The C Programming Language, MoreAwards: Turing Award, National Medal of Technology

and Innovation, IEEE Emanuel R. Piore Award

UNIX Operating SystemAuthors of UNIXKen ThompsonComputer DesignerKenneth Lane "Ken" Thompson, commonly referred to as ken in hacker circles, is an American pioneer of computer science. Having worked at Bell Labs for most of his career, Thompson designed and implemented the original Unix operating system. Born: February 4, 1943 (age 72), New Orleans, LAFields: Computer ScienceAwards: Turing Award, National Medal of Technology and Innovation,Tsutomu Kanai Award, IEEE Emanuel R. Piore AwardEducation: University of California, Berkeley (1966), University of California, Berkeley (1965)

UNIX Operating SystemWhat is the next user interface going to be?

The textual (command line) and the visual (graphical user) interfaces are the two most common modalities used to support engineers in network and system administration positions. The command line interface is recognized as the first generation and the graphical user interface is considered the second generation. Currently, research is trying to determine the next best interface. The command line interface is known as, “under the hood” method of interacting with the operating system.

UNIX Operating SystemCLI Benefits

Manipulate textual dataQuick customization of data allows engineers the ability to change data to another formExcellent for filtering data on systemsCommands are rich, expressive, flexible, and powerful

GUI Benefits

Reduces data overloadSimple filtering and manipulation of the dataExcellent for displaying trends in data

UNIX Operating SystemCustomizing the UNIX Shell

The UNIX operating system shell is a user program that is executed by the kernel when you log in to your terminal. As previously displayed, UNIX operating systems usually provide several shells for users to select from. The most common shells are the Bourne Shell (sh), the C Shell (csh) and the Korn Shell (ksh). Each of the shells listed vary in some way for specific reasons. Demonstrate: cat /etc/shells

UNIX Operating SystemFeature Function sh csh ksh

Job control Allows processes to be executed in the background

No Yes Yes

History substitution

Allows previous commands to be saved, edited, and reused

No Yes Yes

File name completion

Allows automatic completion of partially typed file name

No Yes Yes

Command line editing

Allows the use of an editor to modify the command line text

No No Yes

Command aliasing

Allows users to rename commands

No Yes Yes

UNIX Operating SystemChanging shells

It is possible to invoke a different shell, while within another shell, you just type the name of the shell you want to execute after the terminal prompt, ksh, csh, or sh.Command renaming

Both ksh and csh provide command name aliasing, which allows you to change a command name. Command aliasing can save you time and keystrokes by renaming frequently issued long commands. Two pieces of information are needed. The command you want to alias, and the alias you want to use to refer to it.

alias g=‘grep’

UNIX Operating SystemFile name completion

If you dislike lengthy typing, csh and ksh provide a feature the allows you to type partial file names. By typing a partial file name and pressing the ESCAPE key once for csh and twice for ksh, you can retrieve the file.

If the beginning characters for a file are entered and the file does not exist, the shell will beep at you.Command history substitution

Both ksh and csh maintain an ordered list of commands that you issued, allowing them to be retrieved from the list. Typing the history command will displayed the list of previous commands issued.

UNIX Operating SystemSet ksh history command size

HISTSIZE=60; export HISTSIZE

Set csh history command sizeset history=70

Retrieve commands using C ShellRecall the last command

!!Recall the command number nine

!9

UNIX Operating SystemUNIX File System

A disk drive is a device that stores data by making electrical imprints on a magnetic surface, optical, or mechanical changes to a surface layer of one or more rotating disks.

A hard disk consists of one or more circular aluminum platters of which either or both surfaces are coated with a magnetic substance used for recording the data. For each surface, there is a read-write head that examines or alters the recorded data. The platters rotate on a common axis; typical rotation speed is 5400 or 7200 rotations per minute, although high-performance hard disks have higher speeds and older disks may have lower speeds. The heads move along the radius of the platters; this movement combined with the rotation of the platters allows the head to access all parts of the surfaces.

UNIX Operating System

UNIX Operating System Hard Drive

UNIX Operating System

The Power of MemoryKeeping data in memory is the current

direction technology is headed instead of pulling data in from a disk during processing. Database companies are interested in getting their share of the in-memory database market. Industries actively involved are Aerospike, Microsoft Research, IBM, Oracle, VoltDB, and MemSQL.

UNIX Operating System

The Power of MemoryIn-memory databases is all so know as

main memory databases. The in-memory databases speed up processing in two basic areas.•Data available in memory•The time lag caused by fetching the data from a disk

UNIX Operating System

The Power of MemoryA more basic delay retrieving data from

disk is the way data is stored on disk in blocks. To get to a particular piece of data, the computer must import the whole block from disk, decode it, run the process on the piece it wants, re-encode the blocks, and send the block back where it came from.

UNIX Operating SystemThe Power of Memory

When you store data in memory, you eliminate the overhead associated with block data. When data resides in-memory, you can be much more efficient.

One database vendor sees in-memory databases as the wave of the future for online transactions. They envision the in-memory database handling a third of the database world.

UNIX Operating SystemHow a UNIX file system functions

Every item in a UNIX file system can be identified to belong to one of the four types:•Ordinary files contain text, data, or program information.

They cannot contain another file or directory. They can be viewed as a one-dimensional array.•Directory contain files, and other directories. A directory is

a file that has one line for each item contained within the directory. Each line in a directory file contains the name of the item, and a numeric reference to the location of the item. The numerical reference is called, i-node, which will be discussed at a later time.

UNIX Operating SystemHow a UNIX file system functions

Every item in a UNIX file system can be identified to belong to one of the four types:• Link is a pointer to another file. As a review, a directory is

a list of the names and i-numbers of files. A directory entry can be a Hard Link or a Symbolic link. Both of these topics will be addressed at a later time. Special files are files representing input/output (I/O) devices. A special file can be a printer, a disk drive, or a terminal. Because UNIX treats all such devices as a file, it achieves a greater degree of compatibility between I/O devices and I/O of ordinary files, providing more efficient use of software.

UNIX Operating SystemTypes of File and Directory AccessAccess File Meaning Directory Meaning r View file contents Search directory contents

w Alter file contents Alter directory contents

x Run executable file Make your current directory-rwx------ Owner (columns 2-4) 700 (111000000)----rwx--- Group (columns 5-7) 070 (000111000)-------rwx Other (columns 8-10) 007 (000000111)

UNIX Operating SystemFile Descriptors

A file descriptor is generally an index for an entry in a kernel-resident data structure that contains information on all open files. Each process on the system has its own file descriptor table. A user application passes the abstract key to the kernel through a system call, and the kernel accesses the file for the application.

UNIX Operating System

Keyboard

DisplayScreen

Program

#0 stdin

#1 stdout

#2 stderr

File Descriptor

UNIX Operating Systemwc `ls` commands

The word count (wc) command reads either standard input or a list of files and generates one or more of the following statistics: newline count, word count, and byte count.

A snippet of output from the wc –l `ls` command:

19 xyz.sh2 zombie24 zombie.c9890 total

UNIX Operating SystemShell and Programs Access

To run a shell script, you will need read (r) and execute (x) access (r-x). The read access mode is a binary 4. The execute access mode is a binary 1.

To run a binary executable program, you will need execute (x) access (--x). The execute access mode is a binary 1.

UNIX Operating System

Character Action> Redirect standard output

>& Redirect standard output and standard error

>>& Append standard output and standard error

< Redirect standard input

UNIX Operating SystemShell redirect

ls –a > /tmp/output 2>&1 (stderr redirected to stdout)

> is equivalent to 1> (stdout (write) redirect symbol))

< is equivalent to <0 (stdin (read) redirect symbol)

UNIX Operating System

Pipe command (command1 | command2)

Command2 does not start executing until command1 has finished, and a sufficiently large scratch file is required to hold the intermediate results as well as whatever work space each task required. 

command1 > tempfile command2 < tempfile rm tempfile

UNIX Operating System

Pipe command

UNIX Operating SystemWhat is a data structure?

A data structure is a specific way of storing and organizing data in a computer so that it can be accessed with high efficiently. Data structures can be used as a single place for storing unrelated information. A data structure is a particular way of storing and organizing data in a computer so that it can be used efficiently.Different kinds of data structures are suited to different kinds of applications, and some are highly specialized to specific tasks.

Some common data structures: array, hash table, linked list, queue, and stack.

UNIX Operating SystemWhat is a shared memory?

Shared memory is memory that may be simultaneously accessed by multiple programs with an intent to provide communication among them or avoid redundant copies. Shared memory is an efficient means of passing data between programs. Depending on context, programs may run on a single processor or on multiple separate processors.

UNIX Operating SystemWhat shared memory looks like?

UNIX Operating SystemWhat shared memory looks like?

UNIX Operating SystemWhat shared memory looks like?A shared memory arena file is a regular or shared

memory device file /dev/shm is mapped and shared by one or more processes.

UNIX Operating System

What is a regular expression?A regular expression (abbreviated regex or

regexp) is a sequence of characters that forms a search pattern, mainly for use in pattern matching with strings, or string matching, i.e. "find and replace"-like operations. The concept arose in the 1950s, when the American mathematician Stephen Kleene formalized the description of a regular language, and came into common use with the Unix text processing utilities ed, an editor, and grep (global regular expression print), a filter.

UNIX Operating SystemIntroduction to PerlPerl - Practical Extraction and Report LanguageOriginally developed by Larry Wall, a linguist.Perl is 21 years old and Perl 5 is 14 years old.

Perl is a simple language- Compiles and executes like a shell script or a batch file- Perl doesn’t impose special growth limitations on arrays and data strings- Perl is a composite of C, AWK, and Basic- Originally developed to process text and automating tasks

UNIX Operating SystemPerl’s range of flexibility

System administrationWeb developmentNetwork programmingGUI development

Major featuresProcedural Programming

Sequence or unstructured statementsIncludes routines, subroutines, methods, or functions

Object Oriented ProgrammingModule uses “objects” and their interactions to design applications and computer programs.

UNIX Operating System

Major features (continued)Powerful built-in support for text

processing Large collection of third-party

modules.Variable

A variable is a storage location that can hold any of various kinds of value, as a program sees fit. This storage location is an abstract area known as namespaces.

UNIX Operating System

Perl languagePerl is a free-form language, but is not free of

form. Perl is a free-form language in which you can put spaces, tabs, and new lines anywhere you like, except where they aren’t allowed.

Whitespace cannot be placed within a token. A token is a sequence of characters with a unit of meaning, like a word in the English language. But unlike the typical word, a token can contain other letters and characters as long as they are kept together.

UNIX Operating SystemScope of a Perl variable

The scope of a variable means how far away you can see a variable, looking through one. Perl has two visible mechanisms.Dynamic scoping of local variables, meaning that the

rest of the block, and any subroutines that are called by the rest of the block, can see the variables that are local to the block.

Lexical scoping of “my” variables, meaning that the rest of the block can see the variable, but other subroutines called by the block cannot see the variable.

UNIX Operating System

Variable and constantA variable is a named object that resides in RAM

memory and is capable of being examined and modified. A variable is used to hold information critical to the operation in an embedded system.

A constant is a named object that resides in memory (usually in ROM) and is only capable of being examined.

UNIX Operating SystemAWK and Perl

The AWK utility is an interpreted programming language typically used as a data extraction and reporting tool. It is a standard feature of most Unix-like operating systems.

AWK was created at Bell Labs in the 1970s, and its name is derived from the family names of authors – Alfred Aho, Peter Weinberger, and Brian Kernighan.

The power, terseness, and limits of early AWK programs inspired Larry Wall to write Perl just as a new, more powerful POSIX AWK and gawk (GNU AWK) were being defined.

UNIX Operating SystemAWK Alfred AhoComputer ScientistAlfred Vaino Aho is a Canadian computer scientist best known for his work on programming languages, compilers, and related algorithms, and his textbooks on the art and science of computer programming. Born: August 9, 1941 (age 74), Timmins, CanadaDoctoral advisor: John HopcroftResidence: United States of AmericaEducation: University of Toronto, Princeton UniversityAwards: IEEE John von Neumann Medal

UNIX Operating SystemAWK Brian KernighanComputer ScientistBrian Wilson Kernighan is a Canadian computer scientist who worked at Bell Labs alongside Unix creators Ken Thompson and Dennis Ritchie and contributed to the development of Unix. He is also coauthor of the AWK and AMPL programming languages. Born: January 1, 1942 (age 73), Toronto, CanadaFields: Computer ScienceEducation: Princeton University, University of Toronto

UNIX Operating SystemAWK Peter J. WeinbergerComputer ScientistPeter Jay Weinberger is a computer scientist best known for his early work at Bell Labs. He now works at Google. Weinberger was an undergraduate at Swarthmore College, graduating in 1964. Born: August 6, 1942 (age 73), New York City, NYKnown for: AWKDoctoral advisor: Derrick Henry LehmerEducation: Swarthmore College, University of California, BerkeleyFields: Number theory, Computer Science

UNIX Operating System

Why is awk language so important?

Awk language is an excellent filter and report writer. Many UNIX utilities generate rows and columns of information. Awk is an excellent tool for processing rows and columns, and it is easier to use awk than other conventional programming languages. Perl recognized the importance of awk, so it was included and enhanced in Perl.Example: awk '{print $1}'

UNIX Operating SystemPoints of interestwho | sort > /tmp/test_file.txt

The output of the who command is piped to thesort function and written in ascending order in the/tmp/test_file.txt file. The “who” and “sort” commands both write and read an intermediate file.Test one liners:

whowho | sortwho | sort | /tmp/test_file.txt

UNIX Operating SystemPoints of interest

Knoppix software was designed to be used as a Live CD because of specific features that make it’s performance and stability very suitable. It has been noted in several articles that Knoppix works best from a Live CD. Knoppix enthusiasts have attempted to install this software on a hard disk and encountered problems in the process. These problems are most pronounced when installing updates and new software.

UNIX Operating SystemDemonstrate

The Advanced Scripting lab assignment requires two shell scripts to be written. srch.sh srchfile.sh

Execution of srch.sh and srchfile.sh Case #1: ./srch.sh <pattern> <file name / directory

name>Case #2: ./srch.sh <pattern> <.>The srch.sh script will call the srchfile.sh script to perform a specific task. The srchfile.sh searches for a file with a pattern and outputs the matching information to standard output. After all directory entries have been read, control is returned to the main script, srch.sh.

UNIX Operating SystemMoving Around in UNIX

historywho –runame –ncatchgrpchownhostnamekill

Break-out problems

1. scalar@ARGV2. $ARGV[0]3. filter4. unless5. $NUMBER6. exit 17. $ARGV[1]8. % (%directory)9. $ ($quote)10. @ (@names)11. Regular expressions12. tr [a-z] [A-Z] < foobar > /tmp/foo

Hands-On-InformationLab Assignment 2-1, Simple Shell Scripting, due

January 24, 2016.Lab Assignment 3-1, Advanced Scripting, due

January 30, 2016.Read Chapters 3 and 4 in Essential System

Administration text.Read Module Two listed under the course Web site.Everyone should have received a Shell Quick

Reference document and script logic for Lab Assignment 2-1.

After Class Assistance

Questions?Comments?Concerns?

After each Franklin Live session, I will remain on the session to provide assistance unless otherwise indicated.

Lab assistance, questions, and chat time