SE201: Linear Algebra

Hyosang KangSchool of Undergraduate Studies

DGIST

Spring, 2019

E-mail: hyosang@dgist.ac.kr Class Room: E7-L22(Lec), E3-318(Recit) *Office: Consillience Hall(E7) G11 Class Hours: (Lecture) Mon, Thr 9:00 - 10:30Office Hours: Mon 10:30 - 12:00 (Recitation) Thr 15:00 - 16:00 *Tel: 053-785-6640Web: klein.dgist.ac.krGitHub: github.com/HyosangKang

*Actual hour and classroom may change

Course Description

This is an introductory course on linear algebra. Linear algebra studies systems of linear equa-tions and properties of vectors and matrices, and is useful in natural science and engineering,economics, social science, and especially in machine learning and optimization. I will introducebasic concepts and properties in linear algebra and matrix theory, as well as how to computematrix algebra by Python programming. As an application, I will explain how neural networkand natural language processing(NLP) works via word2vec algorithm.

Course Objectives

This course expects all students to fully understand the material covered in the class. At the endof class, successful students will

1. understand basic concepts in linear algebra;

2. have mathematical proficiency. That is, students are able to write and communicate math-ematics in a higher level;

3. be able to use Python to compute matrix computation;

4. understand the neural network, and be able to construct a mathematical model using it;

5. be able to initiate their own researches on NLP.

1

Spring 2019 SE201 Syllabus

Prerequisites and Texts

There is no formal prerequisite for this course. However, it may help to understand contentscovered in Differential Equations and Applied Calculus(SE101) and Multivariable Caclulus(SE102).Our official textbook is

Strang, Introduction to Linear Algebra, (4th edition or higher)

Useful references are

1. Lay, Linear Algebra and Its Applications, 5th Edition

2. 히라오카 카즈유키, 호리 켄, 프로그래머를위한선형대수

3. 사이토 고키, 밑바닥부터시작하는딥러닝(Deep Learning from Scratch)

Assessments and Grading

The grades are based on two exams(midterm and final), homeworks, research project on NLP,and class participation. Percentages are the following.

• Midterm (30%)

• Final (30%)

• Homeworks (15%)

• Project (15%)

• Class participation (10%)

The table below shows the schematic of the final grade. If the total score is greater than or equalto 90, then the final grade is A+. If the total score is strictly less than 90 and greater than or equalto 80, the final grade will be A0, and so on.

Total Score 90 80 75 65 55 50 40 30 25 15 5 0Grade A+ A0 A- B+ B0 B- C+ C0 C- D+ D0 D-

Exams

There are two exams in this course. Midterm and final will be on 8th and 16th weeks respectively.Exact date,time, and place for exams will be announced 1-2 weeks ahead. The content of the examwill be different from the other sections of the same course..

The exam evaluate how students understand the materials covered in the class. Students mustwrite as detail as possible in their solution. There will be questions on computation, as well asquestions on explaining mathematical concepts. Every solution must be written in full sentenceswith valid logic. There might be a questions which asks students to collaborate with each othersduring the exam. In such circumstance, specific instruction will be provided in the exam.

Midterm and Final are the crucial components for the final grade. If you do not take midtermor final, you will fail the course. Based on 5-point scoring system, each question will be gradedby the following scheme.

2/6

Spring 2019 SE201 Syllabus

• (5 points) The solution is written by complete sentences and delivers correct answer withflawless mathematical ideas. The solution contains detail explanation of the how the answeris obtained.

• (4 points) The solution is written by complete sentences and uses correct ideas and theo-rems. However, it misses a few detail or makes small mistakes. The solution may containsome flaw or logical gaps, which are not crucial part of the answer.

• (3 points) Student explains their work in detail, but uses a wrong/incorrect idea/theory.Or they come up with partially wrong answer. Otherwise, the conclusion of the solution iscorrect, but it has no detail explanations.

• (2 points) There is no detail explanation, nor conclusion/answer in the solution. Studentsvaguely show their ideas, which are not entirely wrong.

• (1 point) The solution contains no complete sentences, or logical explanations. However, itcontains a partially correct statement of the theorems relevant to the problem.

• (0 point) Students attempt to use completely wrong idea and false statement. Solutioncontains no logical statement at all.

Homework

There will be 9 homeworks. The homework consists of mathematical proof and computation,as well as Python programming exercises. The homework should be submitted through LMS.You must scan your work in pdf format and upload the file to designated website. Due dates arewritten in the “Class Schedule” below. Deadlines are 23:59 of each due date. When the homeworkis submitted late, 50% of the total points will be deducted per day. This means that if you late bytwo days, your homework will not be graded.

Project

Each student will initiate a project on NLP individually or within a group. Each group mustchoose their own subject of the research, which must be original. Students must use Python. (Noother tools are allowed.)

The evaluation is based on presentation. During the 10-minute presentation, students mustexplain the research subject, questions and problems, and methods they used for solving theproblem. The following is the grading scheme.

• (Originality) The work must contains an original interpretation / conclusion.

• (Effectiveness) The research have an impact to other research area or real life.

• (Mathematics) The plan and methods of research must be logical and based on concretemathematical theories and models.

3/6

Spring 2019 SE201 Syllabus

Class Participation

The class participation is evaluated by number attendances in class. Each time a student misses,or late a class, 1% of the grade will be lowered. If you miss or late to class more than 10 times,you will fail the course. For exemption, you must provide an official documents (such as adoctor’s note, or an organization’s official letter, etc)

Class Schedule

There are two 75-minute lectures and a 50-minute recitation session each week. In the lecture,we will learn mathematical backgrounds on linear algebra. In recitation session, we will learnpython programming and backgrounds on NLP. You are expected to be well-prepared beforeattending the class. The lecture will be delivered in both English and Korean. The first half of thelecture will be delivered in English and recorded. The second half of the lecture will be deliveredin Korean and will not be recorded. Below is the weekly schedule of the class, but it is subject tochange.

Week 01, 02/25 - 03/01

• (Lecture) Introduction to vectors and linear equations, elimination by matrices (§1.1 – §2.3)

• (Recitation) Introduction to Python programming (Installing Python IDE)

Week 02, 03/04 - 03/08

• (Lecture) Rules for matrix operations, inverse matrices (§2.4 – §2.5)

• (Recitation) Introduction to Python package numpy, matplotlib

Week 03, 03/11 - 03/15

• (Lecture) LU decomposition, transpose and permutations (§2.6 – §2.7)

• (Recitation) Computing matrix algebra in Python

• Homework 1 due 3/15

Week 04, 03/18 - 03/22

• (Lecture) Spaces of vectors, the null space (§3.1 – §3.2)

• (Recitation) File IO, image, word processings in Python

• Homework 2 due 3/22

4/6

Spring 2019 SE201 Syllabus

Week 05, 03/25 - 03/29

• (Lecture) The rank of matrix, the complete solution of linear equations (§3.3 – §3.4)

• (Recitation) Introduction to neural network and machine learning

• Homework 3 due 3/29

Week 06, 04/01 - 04/05

• (Lecture) Independence, basis, dimension, and the four subspaces (§3.5 – §3.6)

• (Recitation) Introduction to NLP: word2vec

• Homework 4 due 4/5

Week 07, 04/08 - 04/12

• (Lecture) Orthogonality, projections, least squares approximations (§4.1 – §4.3)

• (Recitation) Introduction to Python package gensim

Week 08, 04/15 - 04/19 Midterm Exam

Week 09, 04/22 - 04/26

• (Lecture) Orthogonal basis, Gram–Schmidt (§4.4)

• (Recitation) Perceptrons, activation functions

Week 10, 04/29 - 05/03

• (Lecture) Determinants, permutation, and cofactors (§5.1 – §5.2)

• (Recitation) Layers, foward propagation

• Homework 5 due 5/3

Week 11, 05/06 - 05/10 Substitute holiday on 5/6

• (Lecture) Cramer’s rule, inverse, and volume (§5.3)

• (Recitation) Machine learning by gradient descent method

• Homework 6 due 5/10

Week 12, 05/13 - 05/17

• (Lecture) Eigenvalues, diagonalization (§6.1 – §6.2)

• (Recitation) Constructing neural network model with MNIST dataset

• Homework 7 due 5/17

5/6

Spring 2019 SE201 Syllabus

Week 13, 05/20 - 05/24

• (Lecture) Symmetric matrices, positive-definite matrices, and similar matrices (§6.4 – §6.6)

• (Recitation) Project presentation I

• Homework 8 due 5/24

Week 14, 05/27 - 05/31

• (Lecture) Singular value decomposition (SVD) (§6.7)

• (Recitation) Project presentation II

• Homework 9 due 5/31

Week 15, 06/03 - 06/07 Memorial day on 6/6

• (Lecture) Principal component analysis (PCA)

• (Recitation) Project presentation III

Week 16, 06/10 - 06/14 Final Exam

Course Policies

Academic Integrity

You are expected to maintain the highest honor of their action during the semester. You arealso expected to show respects to instructor, class-assistants, and fellow students. If you showsfollowing misconducts during the semester, you will fail the course.

• cheating in exam

• plagiarism

• forging an official documents

In such cases, you will also be reported to the dean of academy for further disciplinary actions.

Missing Exams

If you miss midterm or final, you will fail the course. In the case of emergency such as hospital-ization or attending a family funeral, you must report to the instructor immediately, so that youcan get a make-up exam without penalty.

6/6