System Architecture

Week 1

Technologies are the techies' favourite topic

Do technologies deliver by themselves?

Do all the hyped technologies solved the problems they promise they will solve?

Guess not...

seems even less likely...

nuff said...

Architecture when there is no architecture

Follows usually the Layer Architecture Pattern

Layer Architecture Pattern

MVC!

Hold on

Is MVC an Architectural Pattern?

Answer•It is not. From Wiki: "Model–view–controller (MVC) is a software design pattern for implementing user interfaces on computers."•You can have MVC in monoliths, micro-services, event-based systems, etc.

 Layers• Components are organized in horizontal layers, with each layer

performing a specific role in the application (e.g. presentation, business, persistence).

• Each layer has a unique responsibility within the system.

• Each layer only needs to know how to talk with the layers adjacent to it and perform the tasks it is meant for.

Components• Components within a layer materialize logic that provides

functionality within the layer (e.g. controllers in a MVC application).

• The layer specifies the component scope, making it easy to develop, test, govern and maintain the application.

• Do language or paradigm-specific constructs (OO-classes, class hierarchies, FP-higher order functions, function modules,…) play any role in our description?

What can go wrong, anti-patternsSinkhole: • Requests pass through multiple layers with little or no logic executed.• In MVC parlance, fat controllers-thin models, vice versa on any similar

combination.• 80-20 is a good rule of thumb to find whether you have a sink hole

(80% of the requests get some processing and 20% only pass through – which makes sense).• In the opposite case, you should may consider making some non-

adjacent layers open.

Can we have less layers?

Or more?

But when 

?

And how we figure it out?

Easy!

Separation of Concerns (Soc – not System-on-Chip)

• Hands down the most important principle in Software Engineering.

• “separation of concerns … even if not perfectly possible is yet the only available technique for effective ordering of one’s thoughts” -- Edsger Dijsktra, 1974

• Invented by him in his 1974 paper 'On the role of scientific thought'.

And who the heck is Dijkstra?• Pioneer in Software Engineering and Architecture, Parallel Computing, Graph

Processing, Programming Paradigms, Algorithms, Compilers,…..

• His minimum cost, graph traversing algorithm from 1959 is still widely used today. He came up with it in 20 minutes, while he was sipping his coffee on a coffee in Amsterdam with his fiance.

• His book 'A Discipline of Programming' is still the best 217 pages of CS material you will ever read.

• His paper 'On the Role of Scientific Thought' is a seminal work in software analysis.

• Early form of hipster.

What is Separation of Concerns?• Separating a computer program in different modules, so that each

module addresses a concern.• A concern is a particular set of information that has an effect on the

code of a computer program.• E.g. Read an entry from a database table and affect the program

workflow.• Process the user input from a form.• Identify an object in an image and decide the course of action.

Excerpts from 'On the Role of Scientific Thought' 

"Let me try to explain to you, what to my taste is characteristic for all intelligent thinking. It is, that one is willing to study in depth an aspect of one's subject matter is isolation for the sake of ist own consistency, all the time knowing that one is occupying oneself only with one of the aspects.

We know that a program must be correct and we can study it from that viewpoint only; we also know that it should be efficient and we can study its efficiency on another day, so to speak. In another modd we may ask ourselves whether, and if so: why, the program is desirable..."

Excerpts from 'On the Role of Scientific Thought' 

"...But nothing is gained –on the contrary!-- by tackling these various aspects simultaneously. It is what I sometimes have called "the separation of concerns", which even if not perfectly possible, is yet the only available technique for effective ordering of one's thoughts, that I know of.

This is what I mean by "focusing one's attention upon some aspect": it does not mean ignoring the other aspects, it is just doing justice to the fact that from this aspect's point of view, the other is irrelevant. It is being one- and multiple-track minded simultaneously."

Excerpts from 'On the Role of Scientific Thought' 

"...The proper technique is clearly to postpone the concerns for general acceptance until you have reached a result of such a quality that it deserves acceptance. It is the significance of your message that should justify the care that you give to ist presentation, it may be uts 'unusualness' that makes extra care necessary.

And secondly what is 'general'? Has Albert Einstein failed because the Theory of Relativity is too difficult for the average highscholl student? "

Excerpts from 'On the Role of Scientific Thought' 

"The task of 'making a thing satisfying our needs' as a single responsibility is split into two parts 'stating the properties of a thing, by virtue of which it would satisfy our needs' and 'making a thing guaranteed to have the stated properties' ".

Business data processing systems are sufficiently complicated to require such a separation of concerns and the suggestion tha in that part of the computing world 'scientific thought is non-applicable luxury' puts the cart before the horse: the mess they are in has been caused by too much unscientific thought."

Do concerns sound familiar?

ActiveSupport::Concernmodule Mailable extend ActiveModel::Conern

def send_password_reset_email UserMailer.password_reset(self).deliver_now end

def send_confirmation_email UserMailer.confirmation(self).deliver_now endend

ActiveSupport::Concern (cont.)• Not plain concerns but cross-cutting concerns.

• Concerns which are spread out and cannot be cleanly decomposed from other concerns.

• Typical examples are logging, indexing.

• Also behavior which does not really seem to be part of the main concern's 'essence' (e.g. password management in a User model).

https://richonrails.com/articles/rails-4-code-concerns-in-active-record-models

The Basic Premise

How can we refactor this code?

The module...

...and the model

SoC can help to build a monolithic application right

What is a monolithic application?

When to build a monolith• When you launch a startup.

• When you have a small development team.

• When you want to iterate fast.

• When scalability is not a (big) issue.

When not to build a monolith• When you grow a startup.

• When you have a big development team.

• When you want to iterate fast and break less things.

• When scalability is not a big issue.

The resuméeAs I hear stories about teams using a microservices architecture, I've noticed a common pattern.

1. Almost all the successful microservice stories have started with a monolith that got too big and was broken up.

2. Almost all the cases where I've heard of a system that was built as a microservice system from scratch, it has ended up in serious trouble.

How far can a Monolith go?

Pretty far actually

kdb+• 19 of the 20 world's top investment banks.

• 1.6 TB daily stream ingestion per server.

• 300 million records/second/core search rate.

• 500KB executable, can fit in the CPUs cache.

Some very successful monoliths

Etsy's Architecture

How many *liths do you see?

Etsy's Architecture (cont.)

Click to add text

Python Interpreter Architecture• The architectural patterns are not limited to one type of application

(web-apps, command line tools, mobile apps), but are mental frameworks which can be used for any software program.

• What is applicable for Etsy, is also applicable for PyPy.

Python Interpreter Architecture (cont.)

Separation of Concerns Revisited

• Did we see SoC being applied to the architectures we saw?

• What could be done better?

Our pet – Wishlist app (vosobe)

A wish list of products

• Organize products you want to buy in lists.

• Share the lists with other persons.

• Follow another user or a user's list.

Languages and Frameworks - #1 Ruby & Rails

Verdict: nayPros:• The current tool of trade.

Cons:• Too much magic (Activerecord, Activeresource).• Devs tend to depend on generators and the Rails magic, don't know what

is happening under the hood and many don't know how to write the code themselves.• Not the cleanest solution architecturally (where to you put your biz logic?)

Candidate Languages - #2 Racket & web-srvr

Verdict: nayPros:• Super interesting language, with cutting-edge features.• Can result in very clean design that follows the intented architecture.• Homoiconicity!

Cons:• It may be too alien for many people, not enough time during

the seminar to get acquained with it.

Candidate Languages - #3 Python & Flask

Verdict: nayPros:• Very simple language and framework.• Can be very flexible and be developed in any direction architecturally.

Cons:• Too simplistic for a real-life application that will need to evolve.• It does not provide enough structure (both good and bad).• Easy to get lost in details if somebody has not enough experiences in

building big projects from scratch.

Candidate Languages - #4 Ruby & Hanami

Verdict: yayPros:• It is in Ruby, simple and elegant language.• It enforces sound architectural choices (real separation of concerns, e.g. models are split in

business logic entities and object that mediates between the entities and the persistence layer). • It is designed to start monolithically and later evolve to microservices. • Performant• Way less magic than rails.• Can expand to multiple applications, while sharing the same entities (models).

Cons:• It may require to write more lines of code than Rails.• Like any other full blown framework, it requires time and effort in order to learn it.

A "Clean" Framework

Vosobe's architecture (looks familiar?)

MySQL

Container

Wishlist (Application)

RepositoriesUserRepository ListRepository ListItem

Repository

ModelsUser List ListItem

Controllers UserController

ListController

ListItemController

ViewsViewLists ViewListItems CreateList AddItemsInList

What about the code?

• Already pushed in Gitlab.

• URL in Google Group page.

Thank you for your time, see you in 2 Fridays :-)