MODULAR SYSTEMS & COMBINATORIAL OPTIMIZATION

(based on course “System design”, 2004…2008, MIPT)

Mark Sh. LevinInst. for Inform. Transmission Problems, Russian Acad. of Sci.

Email: mslevin@acm.org Http://www.mslevin.iitp.ru/

Dept. ‘Technology for Complex System Modeling”,

Division of Applied Mathematics&Informatics, HSE, Moscow, Russia, Dec. 19, 2012

PLAN:1.Preliminaries (about me, about you)

2.Modularity (applications, basic technological problem)3.Decision cycle (problem, model, algorithm, computing, decisions)

4.Combinatorial optimization problems 5.Four-layer framework (basic combinatorial problems/models,

composite models, framework of problems, applied layer)6.Composite problems – research projects

7.Conclusion (about novelty)

PRELIMINARIES

About me: A. Education: (a) Radio Engineering (MTUSI, 1970) (b) Mehmat (MSU,1975)

(c) Faculty of Economics (PhD-studies, MSU)(1981) (d) PhD-engineering & CS (RAS)B. Works: (i) system design, (ii) software engineering: information software packages, (iii)management systems (iv)decision making + COMBINATORIAL OPTIMIZATION

Applications: (a) special systems, (b) governmental organizations (c) manufacturing, (d) geology, (e) house-building, (f) machine-building, (g) communication

C. Recent teaching: Moscow Univs.; MIPT (2004..2008): “System Design”

About you:1.Your future objectives

(i) Work in bank, consulting company, etc. (ii) Establishing new company (Google, Microsoft, Facebook, etc.)

(iii) Academic research & educational work:*BS level: 1...3 conf. papers

*MS level: 1…3 conf. papers, 1…3 journal articles (WoS)PhD level: 3…5 conf. papers, 3…5 journal articles (WoS)

Assistant Prof. level: 8…15 conf. papers, 5…7, journal articles (WoS) Associate Prof. level: 30 journal articles (WoS), 30 conf. papers

Full Prof. level: 60 journal articles (WoS), 60 conf. papers

GOALS: 1.Extending your thinking 2.Possible joint research 3.Usage of my materials

GLANCE

Applications (engineering, IT&CS, economics, geology, biomedicine, etc.)

System approach,

system design,systems

engineering

CombinatorialoptimizationHierarchical

systemmodel

MODULARITY

APPLICATIONS:(1)House-building (2)Computers (3)Machine-building (4)Bioinformatics (5)Software,

(6)Combinatorial chemistry, ETC

BASIC EXAMPLE: linguistics (NB!)

BASIC SYSTEM PROBLEMS: 1. System model (system hierarchy, architecture) 2. System design (system configuration) 3. System improvement/upgrade (adaptation, modification, reconfiguration) 4. Multi-stage system design (design of system trajectory) 5. Combinatorial system evolution (combinatorial modeling) 6. System forecasting

SYSTEM: 1.Basic (e.g., physical) system (e.g., computer, machine/car, house,

communication system, software, algorithm system, personnel) 2.Plan: medical treatment, plan of system improvement, economical plan, rules3.Requirements (system of requirements) 4.Standard (standard as system, system of standard)

COMBINATORIAL OPTIMIZATION PROBLEMS

COMBINATORIAL PROBLEMS (basic problems):There is a set of elements.

Find:Basic Problem 1. Ordering of elementsBasic Problem 2. Grouping of elements

Basic Problem 3. Assignment of elements to certain “places”

PROBLEMS:1.Ordering/scheduling 2.Ranking 3.Knapsack problem 4.Multiple choice problem

5.Assignemnt/allocation/location problems (including marriage problem)6.Clustering (grouping) 7.TSP 8.Graph coloring 9. Covering problems 10.Spanning trees (minimum spanning tree, Steiner tree) 11.SAT etc.

BASIC ISSUES:1.Complexity (Polynomial algorithm exists or does not exist)

2.Design of algorithms: (a) enumerative methods (B-A-B, dynamic programming) (b) polynomial algorithms (c) simple (e.g., greedy) heuristics (c) heuristics

(d) approximation algorithms (e) genetic algorithms (evolutionary optimization)

Multicriteria ranking/choice

Initialset ofalternatives

Linearranking

Choice Ranking

Clustering / Classification Problem

Initial setof objects/alternatives Clusters

Goals:1.To decrease the dimension2.To design a hierarchy

Simple structures (chains, trees, parallel-series graphs)

CHAIN

TREE

PARALLEL-SERIESGRAPH

Simple structures (hierarchy)

Level 4

Level 3

Level 1

Level 2

Optimization problems on graphs: illustrations

a0

a1

a2

a3

a4

a6

a5

a7

a8

a9

BASIC GRAPH (DIGRAPH):weights for arcs (or edges)

2

1

2

2

4

4

1

3 4

3

2

4 3

2

a0

a1

a2

a3

a4

a6

a5

a7

a8

a92

1

2

2

4

4

1

3 4

3

2

4 3

2

Shortest Path for < a0,a9 >:L = < a0,a1,a2,a3,a4,a7,a9 >2+1+1+2+2 = 8

Optimization problems on graphs: illustrations

a0

a1

a2

a3

a4

a6

a5

a7

a8

a9

Spanning tree (length = 19):

2

1

2

4

4

1

3 4

3

2

4 3

2

Traveling Salesman Problem :L = < a0,a1,a3,a5,a7,a9,a8,a4,a2,a6>2+1+3+4+2+2+3+4+4+4

a4

a3 a7

a1

a0 a2

a5 a6a9

a8

a0

a1

a2

a3

a4

a6

a5

a7

a8

a92

1

2

4

4

1

3 4

3

2

4 3

2

2

2

PACKING PROBLEM (illustration)

1

1

2

2

3

3

4

4

5

5

76

6

7

8

8

9

9

10

10

11

11

1213

14 . . .

REGION FORPACKING

ELEMENTS

GOALS:*Maximum of packed elements*Minimum offree space

BIN-PACKING PROBLEM (illustration)

CONTAINERS FORPACKING

ELEMENTS

. . .

1

1

2

2

3

3

4

4

5

5

6

6

GOAL:Usage of minimal number of containers

SCHEDULING: illustrative example for assembly process (algorithm of longest tails)

GOAL:Minimal totalcomplete time

1

3

2

6

5

4

9

8

7 10

11

12

13

14

15

16

17

18

19

Tasks &precedence constraints

1(distancefrom corner)

7

2

23

3

3

4

4

4

55

6

66

6 7

7

7

3 processors:

t

1

2

3

0

17

18

19

12

16

14

13

15

9

10

11

6

7

8

3

4

5 2 1

8

MAXIMA CLIQUE PROBLEM (illustration)

Initial graph G = (R, E), R is set of vertices, E is set of edges

Problem is: Find the maximal (by number of vertices) clique (i.e., complete subgraph)

G = (R,E)

Clique consisting of6 vertices(maximalcomplete subgraph)

Spanning (illustration): 1-connected graph

a0

a1

a2

a3

a4

a6

a5

a7

a8

a9

Steiner tree (example):

2

1

2

4

4

1

3 4

3

2

4 3

2

a4

a3 a7

a1

a0 a2

a5 a6

a9a8

2

a0

a1

a2

a3

a4

a6

a5

a7

a8

a9

Spanning tree (length = 19):

2

1

2

4

4

1

3 4

3

2

4 3

2

a4

a3 a7

a1

a0 a2

a5 a6a9

a8

2

Knapsack problem

max mi=1 ci xi

s.t. mi=1 ai xi b

xi {0, 1}, i = 1, … , m possible additional constraints m

i=1 aik xi bk , k = 1, … , l

. . . . . .

1 i m (index)

a1 ai am (required resource) c1 ci cm (utility / profit)

x1 xi xm (Boolean variable)

Multiple choice problem

max mi=1 qi

j=1 cij xij

s.t. mi=1 qi

j=1 aij xij b

qij=1 xij 1 , i = 1, … , m

xij {0, 1}, i = 1, … , m , j = 1, … , qi

. . . . . .

J1 Ji Jm

. . . . . .

. . .

i | Ji | = qi , j = 1, … , qi

Assignment/Allocation problem

Allocation (assignment, matching, location):

matrix of weights cij

BIPARTITE GRAPH

1

2

3

4

5

6

7

8

a

b

c

d

e

f

g

h

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

a b c d e f g h

1

2

3

4

5

6

7

8

Positions

Set of elements

Assignment/allocation problem

a3

a1

a2

an

b1

FORMULATION (algebraic):Set of elements: A = { a1 , … , ai , … , an }Set of positions: B = { b1 , … , bj , … . bm } (now let n = m)Effectiveness of pair ai and bj is: c ( ai , bj )

xij = 1 if ai is located into position bj and 0 otherwise ( xij { 0,1 } )

The problem is: max ni=1 n

j=1 cij xij

s.t. ni=1 xij = 1 j

nj=1 xij = 1 i

b2

b3

bm

. . . . . .

ELEMENTS POSITIONS

Multiple matching problem

A = { a1, … an } B = { b1, … bm }

C = { c1, … ck }

EXAMPLE:3-MATCHING(3-partitie graph)

Graph coloring problem (illustration)

Initial graph G = (A, E), A is set of vertices, E is set of edges

Problem is: Assign a color for each vertex with minimal number of colors under constraint: neighbor vertices have to have different colors

G = (A,E)

Right coloring

VERTEX COVERING PROBLEM

Vertex set A = { a1, … an }, edge set E={e1, …,ek}, graph G = (A, E)

PROBLEM: find vertex covering (A’ A) that covers e E

Satisfiability problem: illustration for application in software / electronic systems

SYSTEM

x1

xm

xm-1

x2

. . . y (0 or 1)

c1

c2

cn-1

cn

Example: c1 = not x1 OR x2 OR x4 OR not x5 OR x7

c2 = x1 OR not x2 OR not x3 OR x5 OR x7

c3 = not x1 OR not x2 OR x3 OR not x5 OR not xn

c4 = not x2 OR x3 OR x7 OR xn-2 OR xn-1

. . .

y = c1&c2& … &cn

Literal: xi / not xi

PROBLEM: Exist xo=(x1,…,xn) that y(xo) =1 OR not

Satisfiability Problem

SATISFIABILITY

3-SATISFIABILITY

3-MATCHING VERTEX COVERING

PARTITIONING (about knapsack)

HAMILTONIAN CYCLE CLIQUE

BASIC 6 NP-COMPLETE PROBLEMS AND DIAGRAM

Alignment (illustration)

CASE OF 2 WORDS:

A AB B D X

A DA C X Z

Word 1

Word 2

A AB B D X Z

Superstructure

C

A AB B D X

A DA C X Z

Morphological clique

PART 1

PART 2 PART 3

Vertices(design alternatives)

Edges(compatibility)

NOTE:about k-matching

COMPLEXITY OF COMBINATORIAL OPTIMIZAITON PROBLEMS

Polynomial solvableproblems

NP-hardproblems

Approximatepolynomial

solvableProblems(FPTAS)

Knapsackproblem

Multiple choiceproblem

Quadratic assignmentproblem

Morphologicalcliqueproblem

Cliqueproblem

TSP

DECISION CYCLE & Support Components

Applied modularproblem(s)

Solving process(e.g., computing)

DECISION

Frame of problems/mathematical models

Library of problems/ models

Solving schemesAlgorithms

Procedures

Program (software)/interactive procedure

Four-layer Framework

Multicriteriaranking

Knapsackproblem

Multiple choice

problem

Cluste-ring

Assignment/allocation

Design of hierarchy (clustering,

multicriteria spanning)

Morpholo-gical clique

(synthesis)

ETC

Layer 1: Basic problems/models

Multi-criteria

knapsack

Multicriteriamultiple choice

problem

Multi-criteria

assignment/allocation

ETC

Layer 2: Composite models/procedures

Four-stagecomposite

framework

Hierarchicalmorpho-logicaldesign

Systemupgrade/ improve-

ment

Systemevolution/

forecasting

ETC

Layer 3: Basic (typical) solving framework

Design/planning

oftesting

Modulardesign ofsoftware

Informa-tion

retrieval

Design ofmarke-

tingstrategy

ETC

Layer 4: Domain-oriented frameworks

Planningof

mainte-nance

Planningof

medicaltreatment

Improve-ment

ofnetwork

Evolutionof

require-ments

Span-ningtree

Steinertree

problem

Shortestpath

problem

Multistagedesign

S=X*Y*Z

Y Z=P*Q*U*V

Z1=P2*Q3*U1*V5

Z2=P1*Q2*U3*V1Y1

Y2

Y3

A B

A1

A2

A3

B1

B2

B3

B4

C1

C2

C3

C4

C5

D=I*J

I1

I2

I3

J1

J2

J3

J4

P1

P2

P3

Q1

Q2

Q3

Q4

U1

U2

U3

V1

V2

V3

V4

V5

V6

C

X=A*B*C*D

P Q U V

X1=A1*B2*C4*D3

X2=A3*B4*C2*D1

D1=I1*J1

D2=I1*J2

D3=I3*J4

S1=X2*Y3*Z2

S2=X1*Y2*Z1

JI

Illustration for HMMD approach

Compatibility for I,J

Compatibilityfor P,Q,U,V

Compatibilityfor A,B,C,D

Compatibilityfor X,Y,Z

Product trajectory

Stage 1

. . .

T0Stage 3

. . .

Stage 2

. . .

Trajectory

1

2

3

4

a

b

c

d

e

f

g

h

Groups of testers

SYSTEMS

FRAMEWORK: Clustering, Assignment, Multiple Choice Problem

TESTERS

2

Groupsof systems

X

X

XTest actions:A1 (no test)A2 (simple test)A3 (analysis test)A4 (structure research)A5 (new test)

MPEG 1

T0

MPEG 2

MPEG 4

Forecast

Improvement

Evolution as Generations of MPEG-like standard, Forecasting

SYSTEM LIFE CYCLE (product, etc.)

R & Dt

Manufacturing Testing MarketingUtilization &Maintenance Recycling

0 T

T: about 12 years (submarines, airplanes, nuclear technology, etc.) TENDENCY: increasing T (2 years, 6 months)

SYSTEM: 1.Basic (e.g., physical) system (e.g., computer, machine/car, house,

communication system, software, algorithm system, personnel) 2.Plan: medical treatment, plan of system improvement, economical plan, rules3.Requirements (system of requirements) 4.Standard (standard as system, system of standard)

Car:1.Body2.Motor3.Drive system4.Electronics5.Safety

MY COMBINATORIAL TECHNOLOGICAL SYSTEMS PROBLEMS:1.Design of system model2.System design3.System improvement4.System evolution5.System forecasting

Research Opportunity (with Novelty)

Novelty: new approach at each level/for each component:

1.Applied problem (engineering, CS, economics)

2.Model/Model framework

3.Algorithm / algorithm framework

4.Software (program, program package)

EXAMPLES OF STUDENT RESEARCH PROJECTS:ABOUT 30 PUBLISHED PAPERS (my site)

EXAMPLES OF MY RESEARCH PROJECTS:MY PUBLISHED PAPERS (my site)

That’s All

 Thanks!http://www.mslevin.iitp.ru/

Mark Sh. Levin