Semiotics in Media Frank Nack (ISLA - UvA)
description
Transcript of Semiotics in Media Frank Nack (ISLA - UvA)
![Page 1: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/1.jpg)
Semiotics in Media Frank Nack (ISLA - UvA)
![Page 2: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/2.jpg)
Semiotics - Definition
Semiotics is the study of sign processes (semiosis), signs and symbols, or signification and communication. It is usually divided into the three following branches:
• Semantics: Relation between signs and the things to which they refer
• Syntactics: Relations among signs in formal structures • Pragmatics: Relation between signs and their effects on
the people who use them
![Page 3: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/3.jpg)
A fragment of semiotic history
Image by Göran Sonesson: The semiotic function and the genesis of pictorial meaning, http://www.arthist.lu.se/kultsem/sonesson/ImatraCourseTx1.html
![Page 4: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/4.jpg)
Context of pictorial semiotics
Image by Göran Sonesson: The semiotic function and the genesis of pictorial meaning, http://www.arthist.lu.se/kultsem/sonesson/ImatraCourseTx1.html
![Page 5: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/5.jpg)
Communication – or the creation of meaning
![Page 6: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/6.jpg)
Fundamentals of Communication
Signal Source Signal Observer
p c
p = perceive c = conceive
![Page 7: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/7.jpg)
Communication chain
Expedient Percipient
p c p c
Communication
• is a process of transferring information from one entity to another • is sign-mediated interaction between at least two agents • both agents share a repertoire of signs and semiotic rules.
![Page 8: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/8.jpg)
Communication chain
p c p c
Sign Repertoires
![Page 9: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/9.jpg)
Description chain
p c p p
Reality Description
![Page 10: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/10.jpg)
Query chain
Describer Archiver Enquirer
![Page 11: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/11.jpg)
Communication semantics
Task knowledge
Expedient
structures Shared cultural
Shared social structures
Personality attributes
Organismic attributes e.g. male, adult, etc.
Outside cultural attributes
Outside social attributes
Percipient Media
Expression knowledge
Thematic knowledge
Meaning
![Page 12: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/12.jpg)
Media semiotics
Text Image Video Audio Tactile
![Page 13: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/13.jpg)
Semiotic concepts
![Page 14: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/14.jpg)
14
The Sign - Saussure
Concept
Mental Perception of Media
SIGN
Signifier Signified beauty
![Page 15: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/15.jpg)
15
The Sign - Peirce
SIGN
Representamen (symbol)
Object (referent)
Interpretant (thought)
active process
physical or mental entity?
psychological or ontological status?
referred to on a particular occasion? typical or ideal representation?
![Page 16: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/16.jpg)
16
The Sign - Arbitrariness
Conventional in the Saussurean sense means that the relationship between the signifier and the signified dependents on social and cultural conventions.
The Saussurean model supports the notion of arbitrariness of the sign by proposing the autonomy of language in relation to reality. Its emphasis on internal structures within a sign system assumes that language does not “reflect” reality but rather constructs it.
![Page 17: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/17.jpg)
Semantics – Index, Icon Symbol (Peirce)
Icon A sign which represents its object mainly through its similarity with some properties of the object, based on the reproduction of perceptual conditions.
Symbol A sign with an arbitrary link to its object (the representation is based on convention).
Index A sign which represents its object by an inherent relationship.
![Page 18: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/18.jpg)
Semiotics – Text, Image, Video
![Page 19: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/19.jpg)
Text – a sign system
![Page 20: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/20.jpg)
Approaching text
A (Alphabet)
Saussure, Ferdinand de - (1857-1913) Swiss linguist. His Course in General Linguistics (1916, posthumous) is generally considered to be the foundation of modern linguistics. He envisaged the development of semiology as a science of signs.
Peirce, Charles S. - (1839-1914) American scientist and philosopher. One of the foremost philosophers of 'pragmatism' - no object or concept possesses validity or importance in its own right. Its significance lies only in the practical effects of its use or application. For Communication and Media students, his importance lies primarily in his development of semiotics.
B (Logogram)
![Page 21: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/21.jpg)
Approaching text
![Page 22: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/22.jpg)
Encoding - decoding
Encoding : framework of knowledge relation of production technical infrastructure
Decoding : syntactic recognition of the sign (in relation to other signs); semantic comprehension of the intended meaning of the sign; pragmatic interpretation of the sign in terms of relevance, agreement etc.
dominant reading
negotiated reading
oppositional reading
p c p c Sign
Text – a sign system I
![Page 23: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/23.jpg)
Text – a sign system II
Syntagms are often defined as 'sequential' (and thus temporal - as in speech and music), but they can represent spatial relationships. The plane of the syntagm is that of the combination of 'this-and-this-and-this' (syntax).
A paradigmatic structure represents potential substitutions in which a range of candidates can take the place of a sign in the syntagmatic structure. The plane of the paradigm is that of the selection of 'this-or-this-or-this' (semantics).
shoes socks pants sweater scarf hat
knickers short
kilt tights
Example:
![Page 24: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/24.jpg)
Text – a sign system III
Syntagm • Spatial relations (horizontal and vertical axi, centre and margin) • Logical order (grammar) • Exposition (proposition, evidence, justification • Narrative space (exposition, retardation, digression, omission, redundancy) • Narrative time (ellipses, compression, insertion, dilation) Paradigm • clusters (e.g. synonyms) • doublets (e.g. oppositions) • proportional series ( e.g. a series of oppositional doublets such as female - male, passive - active, etc.) => Taxonomy • hierarchies (ordered semantic units based on relations of inclusion or exclusion, e.g. Pekinese/dog/animal/living thing). => Thesaurus
Semantic field: '...a conceptual structure which organises potential meanings in relation to others' => Conceptual graph, semantic network, ontology
processes
Representation and Transformation mechanisms
![Page 25: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/25.jpg)
Text – a sign system IV
A code is a rule-governed system of signs, whose rules and conventions are shared amongst members of a culture, and which is used to generate and circulate meanings in and for that culture. Fiske, J. (1987, p.4)
A set of signs that carry meaning.
A set of agreed rules for combining those signs together
Perceptual (e.g. Typography) Syntagmatic (e.g. Grammar) Paradigmatic (e.g. Ontology) Social (e.g. Word use)
![Page 26: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/26.jpg)
Text – a sign system summary I
Text is a sign system strong on arbitrariness, proposing the autonomy of language in relation to reality.
Text emphasis on internal structures and thus does not 'reflect' reality but rather constructs it.
Text is conventional with an emphasis on the types index and symbol.
![Page 27: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/27.jpg)
Text – a sign system summary II
Representing Text in a media-based system:
Conceptual models for: Typography Layout Writing system (e.g. Alphabet) Syntax (e.g. grammar, markup languages, ....) Dictionaries Semantics (e.g. taxonomy, thesaurus, ontology, conceptual graph, etc.) Style (e.g. frame, template, script,....) Genre (e.g. template, conceptual graph)
Interpretation depends on the task: Search (e.g. text understanding, word matching and/or ranking) Generation (e.g. text understanding, question- answering, ....) Comparison (e.g. Syntax (pattern matching) or semantics (clustering, distance evaluation, etc.)
![Page 28: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/28.jpg)
Text – Example use
Twitoems
Syntagma
Syntagma + IPA (code)
Aesthetic codes Rhetorical codes Stylistic codes
Perceptional Codes
![Page 29: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/29.jpg)
29
Image – a sign system
" Legend of Orpheus & Eurydice ", 2001, The Werner Collection http://www.wernercollection.com/WorldView1.htm
![Page 30: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/30.jpg)
Approaching an image
Concept
Mental perception of media
SIGN
Signifier (material)
Signified (meaning) ?
![Page 31: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/31.jpg)
31
Approaching an image
Colour Object
Mise en scene Framing Genre
Meaning
Materiality
Distance (foreground - background)
![Page 32: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/32.jpg)
32
Image – a sign system I
Perceptual codes • perceptive codes (establish the condition for effective perception) • recognition codes which are blocks of signifieds we use to recognize objects • transmission codes which construct the determining conditions for the perception of an image (dots that make up a newspaper image)
Textual codes • tonal codes address the prosodic features by connoting them with particular intonation of the sign • Iconic codes (figures, signs, semes) • Iconographic codes connote more complex and culturalized semes that are immediately identifiable and classifiable, such as "the four horsemen of the Apocalypse".
![Page 33: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/33.jpg)
33
Image – a sign system II
Social codes • verbal language • bodily codes (bodily contact, physical orientation, gaze, gestures and posture); • commodity codes (fashions, clothing, cars); • behavioural codes (protocols, rituals, role-playing, games) • ideological codes (encoding' and 'decoding' information by using theories such as individualism, liberalism, feminism, materialism, capitalism, socialism, etc.)
Syntagmatic - paradigmatic codes • scientific codes, including mathematics; • aesthetic codes (poetry, drama, painting, sculpture, music, etc.) • genre, rhetorical and stylistic codes (e.g. in narrative: plot, character, action, dialogue, setting, etc.), • mass media codes (e.g. in photography, TV, film, radio, newspaper and magazine, etc.)
![Page 34: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/34.jpg)
34
Image – a sign system III
Denotation describes the 'literal' or 'obvious' meaning of a sign. Thus, denotation of a representational visual image is what all viewers from any culture and at any time would recognize the image as depicting.
Denotation is the first level of signification.
Perceptual codes Textual codes Social codes
Sensory system Media
Sign I (denotative sign with signifier and signified)
![Page 35: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/35.jpg)
35
Image – a sign system IV
Connotation refers to the socio-cultural and 'personal' associations (ideological, emotional etc.) of the sign. These are typically related to the interpreter's class, age, gender, ethnicity and so on.
Connotation is the second level of signification. Sign I + Signified
Social codes Syntagmatic codes + experiences Paradigmatic codes associations
Sign II (connotative sign with signifier and signified)
![Page 36: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/36.jpg)
36
Image – a sign system VI
The third level of signification.
Sign III (valued signifier and signified)
Sign II Social codes Syntagmatic codes Paradigmatic codes
value
![Page 37: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/37.jpg)
37
Image – a sign system IV
Signification difference between text and image
On the 1st step: Text => provides an index as a signified Image => sets the signified
The reader replaces each index (word) of the provided order with a signified for his or her liking => the 3rd step of signification does not cause a problem, as it is already matched in the first step
The viewer has to establish the order of importance (using step 2) => particular the 3rd step of signification becomes important as it is the comparison with the own sign system (comparison with what is not shown) that determines how the perceiver values and thus understands the material.
![Page 38: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/38.jpg)
38
Image – Description methods
" Legend of Orpheus & Eurydice ", 2001, The Werner Collection
![Page 39: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/39.jpg)
39
Image – a sign system summary
An image is a a dominantly iconic sign system, proposing a union in relation to reality.
The denotative power of an image, the optical pattern, communicates a precise knowledge, which releases the audience from the process of decision making but leaves a problem of interpretation (signification process).
![Page 40: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/40.jpg)
40
Image – a sign system summary II
Representing an Image in a media-based system:
Conceptual models for: quantitative or qualitative characterization of optical pattern (feature extraction (colour, texture, light, angle, etc.), pattern recognition (line, shape region, etc.), multi-scale signal analysis, ...) Spatial dimensions => textual metadata Semantics (e.g. taxonomy, thesaurus, ontology, etc.) Semantic markers (key word, tag, schema, ....) to express higher semantics , such as forms, styles, genres, aesthetics, social codes.
Interpretation depends on the task: Search (e.g. retrieval by example) Generation (e.g. Qualitative support on features and higher semantics) Presentation (e.g. browsing through collage) Automatic art generation
![Page 41: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/41.jpg)
Image – Example use
Semiotic tagging (collaboration with Ansgar Scherp, West, University Koblenz-Landau)
Icon Index Symbol
Classification of ‘tags’ while annotating
Identifying the meaning of the image in the query based on classification slots
Step A
Step B
![Page 42: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/42.jpg)
Rhetorical tropes
Image from Luis Buñuel’s “Un Chien Andalou”
![Page 43: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/43.jpg)
Rhetorical tropes
A trope describes the use of a word or expression as changed from the original signification to another. Tropes are mainly of four kinds:
Trope
metaphor,
metonymy,
synecdoche,
and irony.
![Page 44: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/44.jpg)
Rhetorical tropes
A metaphor involves one signified acting as a signifier referring to a different signified.
A metaphor is a figure of speech in which an expression is used to refer to something that it does not literally denote in order to suggest a similarity.
Metaphor
![Page 45: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/45.jpg)
Rhetorical tropes
Metonymy
Metonymy is a function which involves using one signified to stand for another signified which is directly related to it or closely associated with it in some way. (Chandler, http://www.aber.ac.uk/media/Documents/S4B/sem07.html)
Metonyms are based on various indexical relationships between signifieds, notably the substitution of
• effect for cause • object for user • substance for form • place for event • place for person • place for institution • institution for people
See also: Catherine R. Langan’s article “Intertextuality in Advertisements for Silk Cut Cigarettes”, available at http://www.aber.ac.uk/media/Students/crl9502.html
![Page 46: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/46.jpg)
Rhetorical tropes
Synecdoche
A synecdoche substitutes a part of a thing for the whole, or the whole for a part, the species for the genus or vice versa.
Examples: • part for whole (fifty sails for fifty ships ) • whole for part (the market for customers) • species for genus - hypernymy (cutthroat for assassin) • genus for species - hyponymy (vehicle for car).
In visual media a close-up is a simple synecdoche - a part representing the whole.
(Jakobson & Halle 1956, p. 92)
Image from Jean-Luc Godard's ‘La Chinoise'
![Page 47: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/47.jpg)
Rhetorical tropes
Irony
Irony involves incongruity between what is expected and what occurs.
Irony involves a shift in modality. The evaluation of the ironic sign requires the retrospective assessment of its modality status. Re-evaluating an apparently literal sign for ironic cues requires reference to perceived intent and to truth status. Thus, an ironic sign is 'double-coded'.
The signifier of the ironic sign seems to signify one thing but another signifier establishes that it actually signifies something very different. Where it means the opposite of what it says (as it usually does) it is based on binary opposition.
![Page 48: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/48.jpg)
48
Video – a temporal visual sign system
![Page 49: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/49.jpg)
Video – Examples for content repurpose
The shining – a romantic comedy
Titanic – The sequel
Schindler's list – the romantic comedy
Mary Poppins – the scary original
All images from YouTube (www.youtube.com)
![Page 50: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/50.jpg)
50
Video – Internal and external context
An image is an index to a story
Images from Stanley Kubrick's 'Shining'
A video is an iconic representation of a story
![Page 51: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/51.jpg)
51
Video – Internal and external context
Media-dependent context
Spatial Temporal
Meta-semantic context
Plot structure Genre References Reviews Personal preferences ....
Images from Stanley Kubrick's 'Shining'
![Page 52: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/52.jpg)
Video – Media-dependent context - temporal
t
Shot context: temporal relationship between frames
Scene context: temporal relationship between shots, e.g. insert
Episode context: temporal relationship between scenes
Montage Metric (absolute length) Rhythmic Tonal
Compression Expansion Insertion Deletion
Images from Stanley Kubrick's 'Shining'
![Page 53: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/53.jpg)
Temporal
Spatial
t
Video – Media-dependent context - summary
All frames from “The Shining” (1980)
Knowledge representation
![Page 54: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/54.jpg)
54
Video – Internal and external context
Media-dependent context
Spatial Temporal
Meta-semantic context
Plot structure Genre References Reviews Personal preferences ....
Images from Stanley Kubrick's 'Shining'
![Page 55: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/55.jpg)
Video – Meta-Semantic context - Narration III
Characteristic objects and actions from the real world
Fabula
Style
Genre a loose set of criteria for a category of composition
Theme as a set of possible narrative objects (comedy, tragedy, etc.)
Plot
Structure Catalyst – Conflict -
Consequence
Images from Stanley Kubrick's 'Shining'
![Page 56: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/56.jpg)
Video – Meta-Semantic context - Narration IV
Images from Stanley Kubrick's 'Shining'
![Page 57: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/57.jpg)
Syntagma in video
Autonomous sequence
Metz's syntagmatic categories (Monaco, 1981, p. 188).
Autonomous shot (establishing shot, insert)
Syntagms
Parallel syntagm (montage of motifs)
Chronological syntagms
Achronological syntagms Bracket syntagm (montage of brief shots)
Descriptive syntagm (sequence describing one moment)
Narrative syntagms
Alternative narrative syntagm (two sequences alternating)
Linear narrative syntagms
Scene (shots implying temporal continuity)
Sequences proper
Episodic sequence (organized discontinuity of shots)
Ordinary sequence (temporal with some compression)
Syntagmatic structures of construction (Monaco, 1981, p. 145).
Space (Frame)
Time (shot, scene, sequence)
Construction Syntagms
![Page 58: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/58.jpg)
Syntagms and paradigms
Paradigm - Examples
shoes socks pants sweater scarf hat
Paradigmatic structures of clothing (Monaco, 1981, p. 341).
skirt knickers short
kilt culottes tights
Choice process example in film: Casting Image from Charlie Kaufman’s ’Synecdoche New York'
![Page 59: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/59.jpg)
59
Video – Context - summary
MPEG - 1 AVI
Video
Time
![Page 60: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/60.jpg)
60
Video – a temporal visual sign system - summary
A video is a compositional unit with individualised semantics.
The semantics may change if a shot is juxtaposed with another shot.
A distinction between filmic (codify the relation to reality) and cinematic codes (codify narrative communication) must be made.
Video, though based on common human content and thematic structures, provides its own realities of time and space which are interwoven in the narrative structure.
A story is a representational system based on two main layers, structure and content, each serving two distinct purposes (form and substance) simultaneously.
![Page 61: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/61.jpg)
Video – Example use
![Page 62: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/62.jpg)
AUTEUR (1996)
• well described video components • generation of humorous sequences from clip database • rules for continuity editing, action generation, themes, and presentation generation
Interactive Production Generator – Narrative Model
Nack (1996)
![Page 63: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/63.jpg)
AUTEUR - Aim
• Automatic composition of visual slapstick sequences.
• Use an existing, arbitrary data base.
• Investigate machine creativity.
![Page 64: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/64.jpg)
AUTEUR – Generation Example I
Motivation
Narrative rules (joke structure) Access via content representation
Editing rules (continuity)
Narrative rules (continuity) Access via content representation
Editing rules (continuity)
![Page 65: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/65.jpg)
Realisation
Narrative rules (joke structure) Access via content representation Editing rules (continuity)
AUTEUR – Generation Example II
![Page 66: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/66.jpg)
Resolution
Narrative rules (joke structure) Access via content representation Editing rules (continuity)
AUTEUR – Generation Example III
![Page 67: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/67.jpg)
Result
AUTEUR – Generation Example IV
![Page 68: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/68.jpg)
Video – Example use
Narranotations
An approach for an association-based story environment, in which a priori unrelated experiences represented in images, are stitched together to guide users through interesting city spaces.
Interesting spaces are described as ‘hypespot’, which facilitates linking the real world with the structure of the story.
Narranotations (annotation to the image) provide information about how an expression can be used as an element within a story.
![Page 69: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/69.jpg)
Video – Example use
N a r r a n o t a t I o n s
![Page 70: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/70.jpg)
Video – Example use
N a r r a n o t a t i o n s
![Page 71: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/71.jpg)
Discussion
Image from Charlie Kaufman’s ’Synecdoche New York'
![Page 72: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/72.jpg)
References
Image from Charlie Kaufman’s ’Synecdoche New York'
![Page 73: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/73.jpg)
Bibliography - Fundamentals
Andersen, P. B. (1997). Semiotic Approaches to Constrution and Assessment Of Computer Systems. Cambridge Universty Press
Barthes, R. Image Music Text. Hill and Wang, New York, 1977.
Bolter, J. G. & Grusin, R. (1999). Remediation - Understanding New Media. Cambridge, Massachusetts: MIT Press.
Chandler, D. (2003). Semiotics for Beginners - http://www.aber.ac.uk/media/Documents/S4B/semiotic.html
Corner, John (1980). 'Codes and Cultural Analysis', Media, Culture and Society 2: 73-86 Davis, H. & Walton P. (Eds.) (1983). Language, Image, Media. Oxford: Basil Blackwell
Derrida, J. Writing and Difference. Routledge & Kegan Paul, London, 1978.
Ducrot, O. and Todorov, T. Encyclopedic Dictionary of the Sciences of Language. The Johns Hopkins University Press, Baltimore, Maryland, 1979.
![Page 74: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/74.jpg)
Bibliography - Fundamentals
Eco, U. A Theory of Semiotics. Indiana University Press, Bloomington, Indiana, 1976.
Fiske, J. (1982). Introduction to Communication Studies. London: Routledge
Greimas, J. (1983). Structural Semantics: An Attempt at a Method. Lincoln: University of Nebraska Press.
Gumpert, G. & Cathcar, R. (1985). Media Grammars, Generations and Media Gaps. Critical Studies in Mass Communication 2: 23-35
Hawkes, T. Structuralism and Semiotics. University of California Press, Berkeley, 1977.
Hodge, R. & Kress, G. (1988): Social Semiotics. Cambridge: Polity
Kendon, A. (1981). Nonverbal Communication, Interaction, and Gesture - Selections from Semiotica. The Hague: Mouton Publishers.
Lévi-Strauss, C. (1968). Structural Anthropology Volume 1, translated by Clair Jacobson, Brooke Grundfest Schoepf. London: Allen Lane.
Lévi-Strauss, C. (1977). Structural Anthropology Volume 2, translated by Monique Layton. London: Allen Lane.
Lippmann, W. (1934). Public Opinion. New York: The MacMillan Company.
![Page 75: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/75.jpg)
Bibliography - Fundamentals
McLuhan, M. The Gutenberg Galaxy: The Making of Typographic Man. University of Toronto Press, Toronto, 1962.
McLuhan, M. (1962). The Gutenberg Galaxy. Toronto: University of Toronto Press
McLuhan, M. & Quentin Fiore (1967). The Medium is the Massage. New York: Bantam
McLuhan, M. (1970). Counterblast. London: Rapp & Whiting
McLuhan, M. Understanding Media: The Extensions of Man. The MIT Press, Cambridge, Massachusetts, 1994.
Metz, C. Film Language: A Semiotics of Cinema. The University of Chicago Press, Chicago, 1974.
Mick, D. G. (1986): 'Consumer Research and Semiotics: Exploring the Morphology of Signs, Symbols and Significance', Journal of Consumer Research 13(2): 196-213
Minsky, M. L. (1988). The Society of mind. London: Picador.
Nöth, W. (1990). Handbook of Semiotics. Bloomington, IN: Indiana University Press
![Page 76: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/76.jpg)
Bibliography - Fundamentals
Packer, R. & Jordan, K. (editors) (2001). multiMEDIA - From Wagner to Virtual Reality. New York, W.W.Norton & Company
Peirce, C. S. (1960). The Collected Papers of Charles Sanders Peirce - 1 Principles of Philosophy and 2 Elements of Logic, Edited by Charles Hartshorne and Paul Weiss. Cambridge, Massachusetts: The Belknap Press of Harvard University Press.
Reddy, M. The Conduit Metaphor: A Case of Frame Conflict in Our Language about Language. in Ortony, A. ed. Metaphor and Thought, Cambridge University Press, New York, 1979, 284-324.
Saussure, F.d. Course in General Linguistics. McGraw-Hill, New York, 1983.
Sebeok, T. A. & Danesi, M. (2000). The Forms of Meaning – Modeling Systsems Theory and Semiotic Analysis, Mouton de Gryter, Berlin
Sowa, J. F. (1984). Conceptual Structures: Information Processing in Mind and Machine. Reading, MA: Addison-Wesley Publishing Company.
Stam, R., Burgoyne, R. and Flitterman-Lewis, S. New Vocabularies in Film Semiotics: Structuralism, Post-Structuralism, and Beyond. Routledge, London, 1992.
![Page 77: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/77.jpg)
Bibliography - Fundamentals
Sturrock, J. (1986): Structuralism. London: Paladin
Todorov, T. Theories of the Symbol. Cornell University Press, 1977.
Wilden, A. (1987). The Rules Are No Game: The Strategy of Communication. London: Routledge & Kegan Paul
Wolff, C. (1972). A Psychology of Gesture. New York: Arno Press.
![Page 78: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/78.jpg)
Bibliography - Text
Ball, M. (1997). Narratology - Introduction to the Theory of Narrative (2nd edition). Toronto: University of Toronto Press.
Barrett, E. (ed.), The Society of Text: Hypertext, Hypermedia, and the Social Construction of Information. The MIT Press, Cambridge, Massachusetts, 1989.
Barrett, E. (ed.), Text, ConText, and HyperText: Writing with and for the Computer. The MIT Press, Cambridge, Massachusetts, 1988.
Barthes, R. Elements of Semiology. Hill and Wang, New York, 1967.
Barthes, R. (1974). S/Z with a preface by Richard Howard. London: Jonathan Cape.
Barthes, R. (1977). Image, Music, Text - Essays selected and translated by Stephen Heath. London: Fontana Press.
Bernstein, M., Patterns of Hypertext. in ACM Hypertext'98, (Pittsburg, Pennsylvania, 1998), ACM Press, 21-29.
Berwick, R.C. Computational Structure of Language.
![Page 79: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/79.jpg)
Bibliography - Text
Berwick, R.C. and Weinberg, A.S. The Grammatical Basis of Linguistic Performance: Language Use and Acquisition. MIT Press, Cambridge, Mass., 1984.
Bordwell, D. (1989). Making Meaning - Inference and Rhetoric in the Interpretation of Cinema. Cambridge, Massachusetts: Harvard University Press.
Brady, M., Berwick, R.C. and Allen, J. Computational Models of Discourse. MIT Press, Cambridge, Massachusetts, 1983.
Brusilovsky, P., Kobsa, A., and Vassileva J.(ed.) Adaptive Hypertext and Hypermedia. Kluwer Academic Publishers, Dordrecht, The Netherlands, 1998.
Bush, V. As We May Think. Atlantic Monthly, 176 (July). 101-108.
Chatman, S.B. Story and Discourse: Narrative Structure in Fiction and Film. Cornell University Press, Ithaca, N.Y., 1978.
Chatman, S.B. Coming to Terms – The Rhetoric of Narrative in Fiction and Film. Cornell University Press, Ithaca, N.Y., 1990.
![Page 80: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/80.jpg)
Bibliography - Text
Chomsky, N. Aspects of the Theory of Syntax. The MIT Press, Cambridge, 1965.
Chomsky, N. Language and Mind. Harcourt Brace & World, Inc, New York, 1968.
Chomsky, N. Studies on Semantics in Generative Grammar. Mouton, The Hague, 1972. Chomsky, N. Rules and Representations, 1980.
Chomsky, N. Lectures on Government and Binding: The Pisa Lectures. Foris Publications, Holland, 1981.
Chomsky, N. The Logical Structure of Linguistic Theory. University of Chicago Press, Chicago, 1985.
Chomsky, N. Language and Thought. Moyer Bell, Wakefield, Rhode Island, 1993.
Chomsky, N. On Language: Chomsky's Classic Works "Language and Responsibility" and "Reflections on Language" in One Volume. The New Press, New York, 1998.
![Page 81: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/81.jpg)
Bibliography - Text
Delany, P. and Landow, G.P. Hypermedia and Literary Studies, 1991.
Eco, U. (1966). 'Narrative Structure in Fleming'. In E. del Buono & Umberto Eco (Eds.): The Bond Affair. London: Macdonald
Eco, U. (1981). The Role of the Reader. London: Hutchinson
Eco, U. (1984). Semiotics and the Philosophy of Language. Bloomington, IN: Indiana University Press
Engelbart, D. A Conceptual Framework for the Augmentation of Human Intellect. in Howerton, P.D., Weeks, D.C ed. Vistas in Information Handling, Spartan Books, Washington DC, 1963.
Goodman, N. Languages of Art. Hackett Publishing Company, Inc., 1976.
Greimas, A.J. Structural Semantics: An Attempt at a Method. University of Nebraska Press, Lincoln, 1983.
Holdcroft, D. (1991). Saussure: Signs, Systems and Arbitrariness. Cambridge: Cambridge University Press
Illich, I. and Sanders, B. ABC: The Alphabetization of the Popular Mind. North Point Press, San Francisco, 1988.
![Page 82: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/82.jpg)
Bibliography - Text
Iser, W. (1989). Prospecting: From Reader Response in Literary Anthropology. Baltimore, Maryland: The John Hopkins University Press.
Iser, W. (1993). The Fictive and the Imaginary: Charting Literary Anthropology. Baltimore, Maryland: The John Hopkins University Press.
Jakobson, R. (1971): 'Language in Relation to Other Communication Systems'. In Roman Jakobson (Ed.): Selected Writings, Vol. 2. Mouton: The Hague, pp. 570-79
Jakobson, R.H., M. Fundamentals of Language. Mouton Publishers, The Hague, 1980.
Kolb, D. Socrates in the Labyrinth: Hypertext, Argument, Philosophy. Eastfgate Systems, Cambridge, 1995.
Kumpf, E.P. Visual Metadiscourse: Designing the Considerate Text. Technical Communication Quarterly, 9 (4). 401-424.
Lakoff, G. P. (1972). Structural complexity in fairy tales. The study of man, 1, 128 - 190.
Lakoff, G. & Johnson, M. (1980). Metaphors We Live By. Chicago: University of Chicago Press
![Page 83: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/83.jpg)
Bibliography - Text
Leiss, W., Kline, S. & Jhally, S. (1990). Social Communication in Advertising: Persons, Products and Images of Well-Being (2nd Edn.). London: Routledge
Lemon, L. T., & Reis, M. J. (1965). Russian Formalist Criticism - Four Essays. Lincoln: University of Nebraska Press.
McQuarrie, E. F. & Mick, D. G. (1999). Visual Rhetoric in Advertising: Text-Interpretative, Experimental and Reader-Response Analyses, Journal of Consumer Research 26(1): 37-54
Miles, A. Hypertext Syntagmas: Cinematic Narration with Links. Journal of Digital Information, 1 (7).
Moulthrop, S., Beyond the Electronic Book: A Critique of Hypertext Rhetoric. in 4th ACM conference on Hypertext, (San Antonio, USA, 1991), 291 - 298.
Moulthrop, S. and Cohen, S. The Color of Television, 1996, A multi-threaded, verbal/graphic Web fiction in collaboration with the writer and designer Sean Cohen.
Nelson, T. Computer Lib and Dream Machines. Microsoft Press, Redmond, Washington, 1987.
Nelson, T. Literary Machines, 1981.
![Page 84: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/84.jpg)
Bibliography - Text
Nielsen, J. Hypertext & Hypermedia. Academic Press, Inc.,, Boston, 1990.
Nielson, J. Multimedia and Hypertext: The Internet and Beyond. Academic Press, Inc, Boston, 1990.
Nyce, J.M. and Kahn, P. (eds.). From Memex to Hypertext: Vannevar Bush and the Mind's Machine. Academic Press, Boston, 1991.
Ong, W. Interfaces of the Word. Cornell University Press, Ithaca, 1977.
Ong, W. The Presence of the Word. Yale University Press, New Haven, 1967.
Ong, W.J. Orality and Literacy: The Technologizing of the Word. Methuen, London, 1982.
Price, G. (1973). A grammar of story: An introduction. The Hague: Mouton.
Propp, V. W. (1968). Morphology of the Folktale. University of Texas Press.
Raskin, V. (1985). Semantic Mechanisms of Humor. Dordrecht: D. Reidel Publishing Company.
Ricoeur, P. (1985). Time and Narrative. Chicago: The University of Chicago Press.
![Page 85: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/85.jpg)
Bibliography - Text
Sampson, G. Writing Systems: A Linguistic Introduction. Stanford University Press, Stanford, California, 1985.
Suleiman, S. R. (1983). Authoritarian Fictions: The Ideological Novel As a Literary Genre. New York: Columbia University Press.
Suls, J. M. (1972). A two-stage model for the appreciation of jokes and cartoons: An information-processing analysis. In J. H. Goldstein & P. E. McGhee (Eds.), The Psychology of Humour (pp. 81 - 100). New York, London: Academic Press.
Saussure, F. d. (1966). Course in General Linguistics - edited by Charles Balley, Albert Sechehaye and Albert Riedlinger. New York: McGraw-Hill.
Segre, C. (1979). Structure and Time - Narration, Poetry, Models. Chicago: The University of Chicago Press.
Tosca, S.P., A Pragmatics of Links. in 11th ACM conference on Hypertext and Hypermedia, (San Antonio, Texas, USA, 2000), pp. 77 - 84.
Winograd, T. Language as a Cognitive Process: Syntax. Addison-Wesley, Massachusetts, 1983.
Winograd, T. Understanding Natural Language. Academic Press, 1972.
![Page 86: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/86.jpg)
Bibliography - Text
Zellweger, P.T., Chang, B-W., and Mackinlay, J. D., Fluid Links for Informed and Incremental Link Transitions. in 9th ACM conference on Hypertext and Hypermedia, (Pittsburgh, USA, 1998), 50 - 57.
![Page 87: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/87.jpg)
Bibliography - Image
Arnheim, R. Art and Visual Perception: A Psychology of the Creative Eye. University of California Press, Berkeley, 1974.
Arnheim, R. (1983). Film as Art. London: Faber & Faber. Argyle, M. (1988): Bodily Communication (2nd edn.). London: Methuen
Aumont, J. The Image. British Film Institute, London, 1997.
Barthes, R. Elements of Semiology. Hill and Wang, New York, 1967.
Barthes, R. Image Music Text. Hill and Wang, New York, 1977.
Barthes, R. S/Z, 1974.
Bazin, A. (1967). The Ontology of the Photographic Image. In H. Gray (Eds.), What is Cinema - essays selected and translated by Hugh Gray (pp. 9 - 16). Berkeley: University of California Press.
![Page 88: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/88.jpg)
Bibliography - Image
Cook, G. (1992). The Discourse of Advertising. London: Routledge
Del Bimbo, A. (1999). "Visual Information Retrieval", Morgan Kaufmann Ed, San Francisco, USA
Eco, U. (1982). Critique of the Image. In Burgin, V. (Ed.), Thinking Photography. London: Macmillan
Forceville, C. (1996). Pictorial Metaphor in Advertising. London: Routledge
Galtung, J. & Ruge, E. (1981). Structuring and Selecting News. In Cohen, S. & Young, J. (Eds.) (1981): The Manufacture of News: Social Problems, Deviance and the Mass Media. London: Constable ., pp. 52-63
Gibson, J. J. (1950). Perception of the Visual World. Boston, Ma: Houghton Mifflin.
Gibson, J. J. (1971). The Information available in Pictures. Leonardo(4), 27 - 35.
![Page 89: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/89.jpg)
Bibliography - Image
Gombrich, E. H (1974). The Visual Image'. In David R Olson (Ed.): Media and Symbols: The Forms of Expression, Communication and Education. Chicago, IL: University of Chicago Press, pp. 255-8; first published in Scientific American 227 (September 1971): 82-96
Gombrich, E. H (1977). Art and Illusion: A Study in the Psychology of Pictorial Representation. London: Phaidon
Gombrich, E.H. Image and Code. Horace H. Rackham School of Graduate Studies, Ann Arbor, MI, 1981.
Gombrich, E. H (1982). The Image and the Eye: Further Studies in the Psychology of Pictorial Representation. London: Phaidon
Gombrich, E. H (1995). The Story of Art. London: Phaidon
Goodman, N. Languages of Art. Hackett Publishing Company, Inc., 1976.
Grayson, K. (1998). The Icons of Consumer Research: Using Signs to Represent Consumers' Reality. In Stern, B. B (Ed) (1998): Representing Consumers: Voices, Views and Visions. London: Routledge, pp. 27-43.
![Page 90: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/90.jpg)
Bibliography - Image
Hall, S. (1981). The Determinations of News Photographs. In Cohen, S. & Young, J. (Eds.) (1981): The Manufacture of News: Social Problems, Deviance and the Mass Media. London: Constable , pp. 226-43
Hochberg, J.E. Perception. Prentice-Hall, New Jersey, 1978.
Kennedy, J.M. A Psychology of Picture Perception. Jossey-Bass, San Francisco, 1974.
Kress, G. & van Leeuwen, T. (1996): Reading Images: The Grammar of Visual Design. London: Routledge
Kress, G. & van Leeuwe, T. (1998): 'Front Pages: (The Critical) Analysis of Newspaper Layout'. In Allan Bell & Peter Garrett (Eds.): Approaches to Media Discourse. Oxford: Blackwell, pp. 186-219
Marr, D. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. Freeman, San Francisco, 1982.
McCloud, S. Understanding Comics: The Invisible Art. Tundra Publishing, Northampton, Massachusetts, 1993.
![Page 91: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/91.jpg)
Bibliography - Image
McLuhan, M. Counter-Blast. Rapp & Whiting, London, 1970.
McLuhan, M. The Gutenberg Galaxy: The Making of Typographic Man. University of Toronto Press, Toronto, 1962.
McLuhan, M. and Fiore, Q. The Medium is the Massage. Bantam Books, New York, 1967.
Panofsky, E. Meaning in the Visual Arts: Papers in and on Art History. Doubleday, Garden City, N.Y., 1955.
Pentland, A.P., Picard, R.W. and Sclaroff, S. Photobook: Tools for Content-Based Manipulation of Image Databases, Massachusetts Institute of Technology Media Laboratory, 1994.
Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A. and Jain, R. Content-Based Image Retrieval at the End of the Early Years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (12). 1349-1380.
Sonesson, G. (1992). The semiotic function and the genesis of pictorial meaning, in Tarasti, Eero, ed.,Center/Periphery in representations and institutions. Proceedings from the 3rd Annual Meeting and Congress of The International Semiotics Insitute, Imatra, Finland, July 16-21, 1990.: International Semiotics Institute, Imatra 1992; ss 211-256.
![Page 92: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/92.jpg)
Bibliography - Image
Strothotte, S.S., T. Seeing Between the Pixels. Springer, Berlin, 1997.
Strothotte, T. Computational Visualization - Graphics, Abstraction and Interactivity. Springer, Berlin, 1998.
Tudor, A. Image and Influence. George Allen & Unwin Ltd., London, 1974.
Tufte, E.R. Envisioning Information. Graphics Press, Cheshire, Connecticut, 1990.
Tufte, E.R. The Visual Display of Quantitative Information. Graphics Press, Cheshire, 1983.
Tufte, E.R. Visual Explanations: Images And Quantities, Evidence And Narrative. Graphics Press, Cheshire, Connecticut, 1977.
![Page 93: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/93.jpg)
Bibliography – Temporal Media
Andrew, J.D. The Major Film Theories: An Introduction. Oxford University Press, Oxford, 1976.
Andrew, D. Concepts in Film Theory. Oxford University Press, Oxford, 1984.
Arnheim, R. Film as Art. Faber & Faber, London, 1983.
Baggaley, J. & Duck S. (1976). Dynamics of Television. Farnborough: Saxon House
Balázs, B. Theory of the Film: Character and Growth of a New Art. Dover Publications, Inc., 1970.
Barthes, R. The Sequences of Actions. in Strelka, J. ed. Patterns of Literary Style, State University of Pennsylvania Press, University Park, Pennsylvania, 1971.
Bazin, A. What is Cinema? (Volume I). University of California Press, Berkeley, 1971.
Bazin, A. What is Cinema? (Volume II), 1971.
![Page 94: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/94.jpg)
Bibliography – Temporal Media
Bellour, R. To Analyze, to Segment. Quarterly Review of Film Studies, 1, 3 (August). 331-354.
Bettetini, G. The Language and Technique of the Film, 1973.
Bloch, G.R. Elements d'une Machine de Montage Pour l'Audio-Visuel, Ecole Nationale Supérieure Des Télécommunications, 1986.
Bloch, G.R. From Concepts to Film Sequences, Yale University Department of Computer Science, 1987.
Bordwell, D., Staiger, J. and Thompson, K. The Classical Hollywood Cinema: Film Style and Mode of Production to 1960. Columbia University Press, New York, 1985.
Bordwell, D. Narration in the Fiction Film. University of Wisconsin Press, Madison, 1985.
Bordwell, D. Making Meaning: Inference and Rhetoric in the Interpretation of Cinema. Harvard University Press, Cambridge, Massachusetts, 1989.
Bordwell, D. The Cinema of Eisenstein. Harvard University Press, Cambridge, Massachusetts, 1993.
![Page 95: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/95.jpg)
Bibliography – Temporal Media
Bordwell, D. On the History of Film Style. Harvard University Press, Cambridge, Massachusetts, 1997.
Bordwell, D. Planet Hong Kong: Popular Cinema and the Art of Entertainment. Harvard University Press, Cambridge, 2000.
Bordwell, D. and Thompson, K. Film Art: An Introduction. McGraw Hill, New York, 2003.
Burch, N. Theory of Film Practice. Princeton University Press, Princeton, 1969.
Buckland, W. (2000). The Cognitive Semiotics of Film. Cambridge, UK: Cambridge University Press.
Carroll, J.M. Toward a Structural Psychology of Cinema. Mouton Publishers, The Hague, 1980.
Chatman, S.B. Story and Discourse: Narrative Structure in Fiction and Film. Cornell University Press, Ithaca, N.Y., 1978.
Davis, D. (1960). The Grammar of Television Production (revised by John Elliot). London: Barrie & Rockliff
![Page 96: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/96.jpg)
Bibliography – Temporal Media
Eco, U. (1965). 'Towards a Semiotic Enquiry into the Television Message', In Corner & Hawthorn (Eds.) (1980), pp. 131-50
Eco, U. Articulations of the Cinematic Code. in Nichols, B. ed. Movies and Methods: An Anthology, University of California Press, Berkeley, 1976, 590-607.
Eco, U. A Theory of Semiotics. Indiana University Press, Bloomington, Indiana, 1976.
Eisenstein, S.M. The Film Sense. Harcourt Brace Jovanovich, Publishers, San Diego, 1947.
Eisenstein, S.M. Film Form: Essays in Film Theory. Harcourt Brace Jovanovich, Publishers, San Diego, 1949.
Eisenstein, S.M. Film Essays and a Lecture. Princeton University Press, Princeton, New Jersey, 1982.
Eisenstein Selected Works: Writings 1922 - 1934. BFI Publishing, London, 1988.
![Page 97: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/97.jpg)
Bibliography – Temporal Media
Eisenstein Selected Works: Towards a Theory of Montage. BFI Publishing, London, 1991.
Fiske, J. (1987). Television Culture. London: Routledge
Fiske, J. & John H. (1978): Reading Television. London: Methuen
Frith, U., & Robson, J. E. (1975). Perceiving the language of film. Perception, 4, 97 - 103.
Greenway, T. and Mouchawar, R., A Visual Language for the Management of Digital Film and Video (Motion Imagery) Archives. in 136th Technical Conference and World Media Expo of the Society of Motion Picture and Television Engineers (SMPTE), (Los Angeles, California, 1994), Society of Motion Picture and Television Engineers, Inc., 1-15.
Gregory, J. R. (1961) Some Psychological Aspects of Motion Picture Montage. Ph.D. Thesis, University of Illinois.
Gregory, R. L. (1971). The Intelligent Eye. London: Weidenfeld&Nicolson.
![Page 98: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/98.jpg)
Bibliography – Temporal Media
Hartley, J. (1996). Popular Reality: Journalism, Modernity, Popular Culture. London: Arnold
Hedges, I. (1991). Breaking the Frame - Film Language and the Experience of Limits. Bloomington: Indiana University Press.
Holtzman, S.R. Digital Mantras - The Languages of Abstract and Virtual Worlds. MIT Press, Cambridge, Massachusetts, 1995.
Isenhour, J.P. The Effects of Context and Order in Film Editing. AV Communications Review, 23 (1). 69-80.
Jenkins, H. Textual Poachers: Television Fans & Participatory Culture. Routledge, New York, 1992.
Jordan, T. H. (1975). The Anatomy Of Cinematic Humor. New York: The Revisionist Press.
Katz, S. D. (1991). Shot by Shot - Visualizing from concept to screen. Stoneham, MA: Michael Wiese Productions in conjunction with Focal Press.
Kracauer, S. (1960). Theory of Film: The Redemption of Physical Reality. New York: Oxford University Press.
![Page 99: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/99.jpg)
Bibliography – Temporal Media
Kuleshov, L. (1974). Kuleshov on Film - Writing of Lev Kuleshov. Berkeley: University of California Press.
Manovich, L. The Language of New Media. The MIT Press, Cambridge, Massachusetts, 2001.
McAleese, R. (1985). Knowledge and Information Mapping and Interactive Video. Aberdeen: University Teaching Centre, University of Aberdeen.
Metz, C. (1974). Film Language: A Semiotic Of The Cinema. New York: Oxford University Press.
Metz, C. (1976a). Current Problems of Film Theory: Mitry's L'Esthétique et Psychologie du Cinema. Vol. II. In B. Nichols (Eds.), Movies and Methods (pp. 568 - 578). Berkeley: University of California Press.
Metz, C. (1976b). On the Notion of Cinematographic Language. In B. Nichols (Eds.), Movies and Methods (pp. 582 - 589). Berkeley: University of California Press.
Metz, C. (1981). Methodological Propositions for the Analysis of Film. In: Eaton, M. (Ed.) Cinema and Semiotics (Screen Reader 2). London: Society for Education in Film and Television, op. cit., pp. 86-98
![Page 100: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/100.jpg)
Bibliography – Temporal Media
Metz, C. (1982). The Imaginary Signifier - Psychoanalysis and the cinema. Bloomington: Indiana University Press.
Monaco, J. How to Read a Film: The World of Movies, Media, and Multimedia: Language, History, Theory. Oxford University Press, New York, 2000.
Neale, S., & Krutnik, F. (1990). Popular Film and Television Comedy. London: Routledge.
Nichols, B. (editor) (1976). Movies and Methods I. Berkeley;University of California Press.
Nichols, B. (editor) (1985). Movies and Methods II. Berkeley;University of California Press.
Oldham, G. First Cut: Conversations with Film Editors. University of California Press, Berkeley, California, 1992.
Packer, R. and Jordan, K. Multimedia: From Wagner to Virtual Reality. Norton, New York, 2001.
![Page 101: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/101.jpg)
Bibliography – Temporal Media
Peterson, J. (1994). Dreams of Chaos, Vision of Order - Understanding the American Avant-garde Cinema. Detroit: Wayne State University Press.
Petric, V. Constructivism in Film. Cambridge University Press, Cambridge, 1987.
Pudovkin, V.I. Film Technique and Film Acting. Bonanza Books, New York, 1949.
Rabinger, M. Directing - Film Techniques and Aesthetics. Focal Press., Boston, 1989.
Reisz, K., & Millar, G. The Technique of Film Editing. Focal/Hastings House, New York, 1969.
Rieser, M. & Zapp, A. (2002). New Screen Media - Cinema/Art/Narrative. London: BFI.
Rosenblum, R. and Karen, R. When The Shooting Stops, The Cutting Begins - A Film Editor's Story. Da Capo Paperback, New York, 1979.
Rush, M. (1999). New Media in Late 20th-Century Art. London:Thames & Hudson
![Page 102: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/102.jpg)
Bibliography – Temporal Media
Salomon, G. and Cohen, A.A. Television Formats, Mastery of Mental Skills, and the Acquisition of Knowledge. Journal of Educational Psychology (69). 612 - 619.
Salt, B. Film Style and Technology: History and Analysis. Starword, London, 1992.
Spottiswode, R. J. (1955). A Grammar Of The Film - an analysis of film technique. London: Faber & Faber.
Stam, R., Burgoyne, R. and Flitterman-Lewis, S. New Vocabularies in Film Semiotics: Structuralism, Post-Structuralism, and Beyond. Routledge, London, 1992.
Weber, K., The Work Practice of Moving Image Indexing. in AAAI-94 Workshop on Indexing and Reuse in Multimedia Systems, (Seattle, Washington, 1994).
Wells, P. Understanding Animation. Routledge, London, 1998.
Woods, W. A. (1985). What's in a link: Foundations for Semantic Networks. In R. J. Brachman & H. J. Levesque (Eds.), Readings in Knowledge Representation (pp. 218 - 241). San Mateo, California: Morgan Kaufmann Publishers.
![Page 103: Semiotics in Media Frank Nack (ISLA - UvA)](https://reader034.fdocuments.us/reader034/viewer/2022042721/568c55991a28ab4916c367f9/html5/thumbnails/103.jpg)
Bibliography – Temporal Media
Wollen, P. Signs and Meaning in the Cinema. Indiana University Press, Indiana, 1972.
Young, B. M (1990): Television Advertising and Children. Oxford: Clarendon Press
Zettl, H. Essentials of Applied Media Aesthetics. in Dorai, C. and Venkatesh, S. eds. Media Computing: Computational Media Aesthetics, Kluwer Academic Publishers, Boston, 2001, 11-38.
Zettl, H. Sight, Sound, Motion: Applied Media Aesthetics. Wadsworth Publishing. Company, Belmont, California, 1999.